Python: how to kill child process(es) when parent dies?

Question

The child process is started with

subprocess.Popen(arg) 

Is there a way to ensure it is killed when parent terminates abnormally? I need this to work both on Windows and Linux. I am aware of this solution for Linux.

Edit:

the requirement of starting a child process with subprocess.Popen(arg) can be relaxed, if a solution exists using a different method of starting a process.

Answer 1

Heh, I was just researching this myself yesterday! Assuming you can't alter the child program:

On Linux, prctl(PR_SET_PDEATHSIG, ...) is probably the only reliable choice. (If it's absolutely necessary that the child process be killed, then you might want to set the death signal to SIGKILL instead of SIGTERM; the code you linked to uses SIGTERM, but the child does have the option of ignoring SIGTERM if it wants to.)

On Windows, the most reliable options is to use a Job object. The idea is that you create a "Job" (a kind of container for processes), then you place the child process into the Job, and you set the magic option that says "when no-one holds a 'handle' for this Job, then kill the processes that are in it". By default, the only 'handle' to the job is the one that your parent process holds, and when the parent process dies, the OS will go through and close all its handles, and then notice that this means there are no open handles for the Job. So then it kills the child, as requested. (If you have multiple child processes, you can assign them all to the same job.) This answer has sample code for doing this, using the win32api module. That code uses CreateProcess to launch the child, instead of subprocess.Popen. The reason is that they need to get a "process handle" for the spawned child, and CreateProcess returns this by default. If you'd rather use subprocess.Popen, then here's an (untested) copy of the code from that answer, that uses subprocess.Popen and OpenProcess instead of CreateProcess:

import subprocess import win32api import win32con import win32job  hJob = win32job.CreateJobObject(None, "") extended_info = win32job.QueryInformationJobObject(hJob, win32job.JobObjectExtendedLimitInformation) extended_info['BasicLimitInformation']['LimitFlags'] = win32job.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE win32job.SetInformationJobObject(hJob, win32job.JobObjectExtendedLimitInformation, extended_info)  child = subprocess.Popen(...) # Convert process id to process handle: perms = win32con.PROCESS_TERMINATE | win32con.PROCESS_SET_QUOTA hProcess = win32api.OpenProcess(perms, False, child.pid)  win32job.AssignProcessToJobObject(hJob, hProcess) 

Technically, there's a tiny race condition here in case the child dies in between the Popen and OpenProcess calls, you can decide whether you want to worry about that.

One downside to using a job object is that when running on Vista or Win7, if your program is launched from the Windows shell (i.e., by clicking on an icon), then there will probably already be a job object assigned and trying to create a new job object will fail. Win8 fixes this (by allowing job objects to be nested), or if your program is run from the command line then it should be fine.

If you can modify the child (e.g., like when using multiprocessing), then probably the best option is to somehow pass the parent's PID to the child (e.g. as a command line argument, or in the args= argument to multiprocessing.Process), and then:

On POSIX: Spawn a thread in the child that just calls os.getppid() occasionally, and if the return value ever stops matching the pid passed in from the parent, then call os._exit(). (This approach is portable to all Unixes, including OS X, while the prctl trick is Linux-specific.)

On Windows: Spawn a thread in the child that uses OpenProcess and os.waitpid. Example using ctypes:

from ctypes import WinDLL, WinError from ctypes.wintypes import DWORD, BOOL, HANDLE # Magic value from http://msdn.microsoft.com/en-us/library/ms684880.aspx SYNCHRONIZE = 0x00100000 kernel32 = WinDLL("kernel32.dll") kernel32.OpenProcess.argtypes = (DWORD, BOOL, DWORD) kernel32.OpenProcess.restype = HANDLE parent_handle = kernel32.OpenProcess(SYNCHRONIZE, False, parent_pid) # Block until parent exits os.waitpid(parent_handle, 0) os._exit(0) 

This avoids any of the possible issues with job objects that I mentioned.

If you want to be really, really sure, then you can combine all these solutions.

Hope that helps!

Answer 2

The Popen object offers the terminate and kill methods.

https://docs.python.org/2/library/subprocess.html#subprocess.Popen.terminate

These send the SIGTERM and SIGKILL signals for you. You can do something akin to the below:

from subprocess import Popen  p = None try:     p = Popen(arg)     # some code here except Exception as ex:     print 'Parent program has exited with the below error:\n{0}'.format(ex)     if p:         p.terminate() 

UPDATE:

You are correct--the above code will not protect against hard-crashing or someone killing your process. In that case you can try wrapping the child process in a class and employ a polling model to watch the parent process. Be aware psutil is non-standard.

import os import psutil  from multiprocessing import Process from time import sleep   class MyProcessAbstraction(object):     def __init__(self, parent_pid, command):         """         @type parent_pid: int         @type command: str         """         self._child = None         self._cmd = command         self._parent = psutil.Process(pid=parent_pid)      def run_child(self):         """         Start a child process by running self._cmd.          Wait until the parent process (self._parent) has died, then kill the          child.         """         print '---- Running command: "%s" ----' % self._cmd         self._child = psutil.Popen(self._cmd)         try:             while self._parent.status == psutil.STATUS_RUNNING:                 sleep(1)         except psutil.NoSuchProcess:             pass         finally:             print '---- Terminating child PID %s ----' % self._child.pid             self._child.terminate()   if __name__ == "__main__":     parent = os.getpid()     child = MyProcessAbstraction(parent, 'ping -t localhost')     child_proc = Process(target=child.run_child)     child_proc.daemon = True     child_proc.start()      print '---- Try killing PID: %s ----' % parent     while True:         sleep(1) 

In this example I run 'ping -t localhost' b/c that will run forever. If you kill the parent process, the child process (the ping command) will also be killed.