How to get exit code from subprocess.Popen?

Question

With the code below, p.returncode is always None. According to the Popen.returncode documentation, this means that the process hasn't finished yet.

Why am I not getting an exit code?

import os
import sys
import subprocess

cmd = ['echo','hello']
p = subprocess.Popen(cmd,
                     stdout=subprocess.PIPE,
                     stderr=subprocess.STDOUT)
try:
    # Filter stdout
    for line in iter(p.stdout.readline, ''):
        sys.stdout.flush()
        # Print status
        print(">>> " + line.rstrip())
        sys.stdout.flush()
except:
    sys.stdout.flush()

print 'RETURN CODE', p.returncode

Please note: The reason why I'm reading each line separately is because I want to filter the output of other, long-running, processes in real-time and halt them based on certain strings.

I'm on Python 2.7.5 (CentOS 7 64-bit).

---

Solution

Thanks to the answer posted by @skyking, I can now successfully capture the exit code like this, using Popen.poll() (Popen.wait() deadlocked my process):

import os
import sys
import subprocess
import time

cmd = ['echo','hello']
p = subprocess.Popen(cmd,
                     stdout=subprocess.PIPE,
                     stderr=subprocess.STDOUT)
try:
    # Filter stdout
    for line in iter(p.stdout.readline, ''):
        sys.stdout.flush()
        # Print status
        print(">>> " + line.rstrip())
        sys.stdout.flush()
except:
    sys.stdout.flush()

# Wait until process terminates (without using p.wait())
while p.poll() is None:
    # Process hasn't exited yet, let's wait some
    time.sleep(0.5)

# Get return code from process
return_code = p.returncode

print 'RETURN CODE', return_code

# Exit with return code from process
sys.exit(return_code)

Answer 1

According to the linked to documentation

The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet.

A negative value -N indicates that the child was terminated by signal N (Unix only).

You have not called poll or wait so returncode won't be set.

On the other hand if you look in to the source code for fx check_output you see that they are directly using the return value from poll to examine the return code. They know that the process has terminated at that point because they have called wait earlier. If you don't know that you would have to call the wait method instead (but note the deadlock possibility noted in the documentation).

Normally the program will have terminated when you've read all of stdout/stderr, but that's not guaranteed and that may be what you're seeing. Either the program or the OS can close stdout (and stderr) before the process actually terminates and then by just calling poll immediately after you've read all the output from the program might fail.