Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Redirect subprocess stderr to stdout

I want to redirect the stderr output of a subprocess to stdout. The constant STDOUT should do that, shouldn't it?

However,

$ python >/dev/null -c 'import subprocess;\
                        subprocess.call(["ls", "/404"],stderr=subprocess.STDOUT)'

does output something. Why is that the case, and how do I get the error message on stdout?

like image 564
phihag Avatar asked Jul 15 '12 21:07

phihag


2 Answers

In Python < v3.5:

A close read of the source code gives the answer. In particular, the documentation is misleading when it says:

subprocess.STDOUT
Special value that (...) indicates that standard error should go into the same handle as standard output.

Since stdout is set to "default" (-1, technically) when stderr=subprocess.STDOUT is evaluated, stderr is set to "default" as well. Unfortunately, this means that stderr output still goes to stderr.

To solve the problem, pass in the stdout file instead of subprocess.STDOUT:

$ python >/dev/null -c 'import subprocess,sys;subprocess.call(["ls", "/404"],
                        stderr=sys.stdout.buffer)'

Or, for compatibility with legacy 2.x versions of Python:

$ python >/dev/null -c 'import subprocess,sys;subprocess.call(["ls", "/404"],
                        stderr=sys.stdout.fileno())'
like image 172
phihag Avatar answered Nov 18 '22 19:11

phihag


Actually, using subprocess.STDOUT does exactly what is stated in the documentation: it redirects stderr to stdout so that e.g.

command = ["/bin/ls", "/tmp", "/notthere"]
process = subprocess.Popen(command, shell=False, bufsize=1, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = ""
while (True):
    # Read line from stdout, break if EOF reached, append line to output
    line = process.stdout.readline()
    line = line.decode()
    if (line == ""): break
    output += line

results in variable output containing the process' output from both stdout and stderr.

stderr=subprocess.STDOUT redirects all stderr output directly to stdout of the calling process, which is a major difference.


EDIT: Updated code for newer Python versions:

command = ["/bin/ls", "/tmp", "/notthere"]
process = subprocess.Popen(command, shell=False, text=True, bufsize=1, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = ""
while (True):
    # Read line from stdout, break if EOF reached, append line to output
    line = process.stdout.readline()
    if (line == ""): break
    output += line
like image 26
Maxxim Avatar answered Nov 18 '22 19:11

Maxxim