Exception handling when using Python's subprocess.Popen

Question

When dealing with open files, Python has the with syntax that makes sure the file closes when leaving the block - regardless of exceptions etc.

with open('foo.txt') as f:
    foo = f.read()

Since processes are resources too, I was wondering: is something similar possible or recommended when using Popen? For example, should Popen.kill(); Popen.communicate() be run in a finally clause - assuming I don't mind blocking until the process finishes?

Answer 1

Starting from Python 3.2 Popen is a context manager.

from the docs:

Popen objects are supported as context managers via the with statement: on exit, standard file descriptors are closed, and the process is waited for.

This should do pretty much what you want.

This is the relevant part from subprocess.py from the standard lib in Python 3.4:

def __enter__(self):
    return self

def __exit__(self, type, value, traceback):
    if self.stdout:
        self.stdout.close()
    if self.stderr:
        self.stderr.close()
    if self.stdin:
        self.stdin.close()
    # Wait for the process to terminate, to avoid zombies.
    self.wait()

Now you can do in Python 2.7

from subprocess import Popen

class MyPopen(Popen):

    def __enter__(self):
        return self

    def __exit__(self, type, value, traceback):
        if self.stdout:
            self.stdout.close()
        if self.stderr:
            self.stderr.close()
        if self.stdin:
            self.stdin.close()
        # Wait for the process to terminate, to avoid zombies.
        self.wait()

if __name__ == '__main__':
    with MyPopen(['ls']) as p:
        print(p)

Answer 2

For 2.7 you could also use the @contextlib.contextmanager:

import contextlib

@contextlib.contextmanager
def manage_process(process):
    try:
        yield process
    finally:
        for stream in [process.stdout, process.stdin, process.stderr]:
            if stream:
                stream.close()
        process.wait()

e.g:

with manage_process(Popen(['ls'])) as p:
    print(p)