Python subprocess module does not return stdout on segfault

Question

I'm running a C executable from Python and this executable sometimes segfaults. When it segfaults the subprocess module does not return anything in stdout or stderr.

Sample code:

import subprocess

proc = subprocess.Popen(['./a.out'], stdin=subprocess.PIPE,
        stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc.communicate()
print 'out: "%s"' % out
print 'err: "%s"' % err
print proc.returncode

The source for a.out is:

int main() {
    printf("hello world\n");
    int x = 1 / 0;
}

The output of ./a.out is:

hello world
Floating point exception

The output of the Python code is (in Linux, python 2.7):

out: ""
err: ""
-8

Is there a way to get the output of the executable even if it crashes?

A generic method to translate returncode to a string message, would also be nice.

Answer 1

The application is almost certainly not flushing its output buffers before it encounters the segmentation fault.

By default, the C stdio library configures buffering for stdout so that the buffers are flushed at every end of line, but only is stdout is a tty. That's why you do see the output when you run a.out from the command line.

You cannot directly make the C program change this from the Python side. But what you can do is give it a tty to use for its stdout, namely a pseudo tty. The details of opening and setting up a pseudo terminal are much more complicated than setting up a regular pipe, but there are some modules that will help you: See Pexpect and this SO question, among others.

Answer 2

Your C program is not flushing its output buffer. The easiest way around this problem is to change the output mode of STDOUT to unbuffered:

#include <stdio.h>
int main(int argc,char **argv) {
    setvbuf(stdout,_IONBF,0,0);
    printf("hello world\n");
    int x = 1 / 0;
}

Now your program (which I edited to fix a typo) produces the correct output:

$ python2.7 x.py
out: "hello world
"
err: ""
0
$

The return code is 0 on my Mac, confusingly, because programs that are terminated for a signal don't have a return code.