Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to handle subprocess.Popen output in both Python 2 and Python 3

I have a simple function that checks if a program is running and returns a boolean as appropriate. It does this by checking the output of the command ps -A using the module subprocess. I'm trying to get this function to work in both Python 2 and Python 3 but am encountering the following error:

TypeError: a bytes-like object is required, not 'str'

How should the function be changed such that it can work in both Python 2 and Python 3?

import subprocess

def running(program):
    results = subprocess.Popen(
        ["ps", "-A"],
        stdout = subprocess.PIPE
    ).communicate()[0].split("\n")
    matches_current = [
        line for line in results if program in line and "defunct" not in line
    ]
    if matches_current:
        return True
    else:
        return False

EDIT: Following some guidance by @Josh, I've changed the newline string delimiter to bytecode, however I'm still encountering a similar problem:

>>> import subprocess
>>> def running(program):
...     results = subprocess.Popen(
...         ["ps", "-A"],
...         stdout = subprocess.PIPE
...     ).communicate()[0].split(b"\n")
...     matches_current = [
...         line for line in results if program in line and "defunct" not in line
...     ]
...     if matches_current:
...         return True
...     else:
...         return False
... 
>>> running("top")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in running
  File "<stdin>", line 7, in <listcomp>
TypeError: a bytes-like object is required, not 'str'
like image 861
BlandCorporation Avatar asked Jan 09 '18 13:01

BlandCorporation


People also ask

How do I get subprocess to Popen output?

popen. To run a process and read all of its output, set the stdout value to PIPE and call communicate(). The above script will wait for the process to complete and then it will display the output.

How do I use subprocess Popen in Python?

The recommended approach to invoking subprocesses is to use the run() function for all use cases it can handle. For more advanced use cases, the underlying Popen interface can be used directly. Run the command described by args. Wait for command to complete, then return a CompletedProcess instance.

What does Python subprocess Popen return?

Popen Function The function should return a pointer to a stream that may be used to read from or write to the pipe while also creating a pipe between the calling application and the executed command. Immediately after starting, the Popen function returns data, and it does not wait for the subprocess to finish.

What is the difference between subprocess run and subprocess Popen?

The main difference is that subprocess. run() executes a command and waits for it to finish, while with subprocess. Popen you can continue doing your stuff while the process finishes and then just repeatedly call Popen. communicate() yourself to pass and receive data to your process.


2 Answers

Use bytestrings like b"\n" instead of just plain "\n", and also use program.encode() instead of program (assuming you're being passed a string):

results = subprocess.Popen(
    ["ps", "-A"],
    stdout = subprocess.PIPE
).communicate()[0].split(b"\n")
matches_current = [
    line for line in results if program.encode() in line and b"defunct" not in line
]

This is because in Python 3, subprocess.Popen.communicate returns bytes rather than a string by default. In Python 2.6+, b"\n" == "\n", so you shouldn't have any problems there.

like image 148
ash Avatar answered Nov 15 '22 05:11

ash


It's an old question, but I've found a simpler way that works both in Python 2 and 3: add .decode() after communicate()[0]:

results = subprocess.Popen(
        ["ps", "-A"],
        stdout = subprocess.PIPE
    ).communicate()[0].decode().split("\n")

So you don't have to add b's before every string literal or encode() to string variables in your code. As a bonus, results will become an Unicode string.

PS. In my case I had subprocess.check_output() and appending .decode() also worked as expected.

EDIT: probably it is better to specify encoding, like decode('utf-8') just in case.

like image 21
akarilimano Avatar answered Nov 15 '22 06:11

akarilimano