pythonpython-3.xdecode

Python decode() mangles bytes object (either returns empty string, or only last line of text)


For the life of me, I can't figure out what's happening here...

I'm capturing the output of a command line utility via subprocess.popen(), and processing the stdout line by line via process.stdout.readline() (which returns a bytes object). I want to convert each stdout line to a string, but when I convert it with output.decode(), it either (1) returns an empty string (even though there is text in the bytes object), or (2) only returns the last line in the bytes object.

I have looked through the python docs for Code.decode, but can't figure out why this is happening or how to remedy it.

Below is a snippet of the code I'm using, and some example outputs.

Code snippet:

output = process.stdout.readline()  # returns a bytes object
if output:
    print(str(output))
    print(output.decode())

Example 1 (only last line being returned by decode()):

first print statement (the bytes object):

b'0M Scan  C:\\Users\\Me\\Documents\\\r                                                                              \r24 folders, 8 files, 65 bytes (1 KiB)'

second print statement (result of decode()):

24 folders, 8 files, 65 bytes (1 KiB)

(I would expect it to be this:)

Scan  C:\\Users\\Me\\Documents\\
24 folders, 8 files, 65 bytes (1 KiB)

Example 2 (decode() returning empty string):

first print statement (the bytes object):

b'0%'

second print statement (result of decode()):

"" (an empty string)

(I would expect it to be this:)

0%

I have tried output.decode('utf-8') but the same result. output is NOT being accessed/modified elsewhere. Why might this be happening? Could this be because I'm on a Windows machine?


Solution

  • For some reason the program is returning \r as the line feeds.

    Please enable text=True on the popen, or add the correct encoding, in order to automatically enable universal newlines.

    Otherwise, replace all \r with \n manually or wrap the stdout with io.TextIOWrapper and newline=\r.