how to kill (or avoid) zombie processes with subprocess module

Question

When I kick off a python script from within another python script using the subprocess module, a zombie process is created when the subprocess "completes". I am unable to kill this subprocess unless I kill my parent python process.

Is there a way to kill the subprocess without killing the parent? I know I can do this by using wait(), but I need to run my script with no_wait().

Answer 1

A zombie process is not a real process; it's just a remaining entry in the process table until the parent process requests the child's return code. The actual process has ended and requires no other resources but said process table entry.

We probably need more information about the processes you run in order to actually help more.

However, in the case that your Python program knows when the child processes have ended (e.g. by reaching the end of the child stdout data), then you can safely call process.wait():

import subprocess  process= subprocess.Popen( ('ls', '-l', '/tmp'), stdout=subprocess.PIPE)  for line in process.stdout:         pass  subprocess.call( ('ps', '-l') ) process.wait() print "after wait" subprocess.call( ('ps', '-l') ) 

Example output:

$ python so2760652.py F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD 0 S   501 21328 21326  0  80   0 -  1574 wait   pts/2    00:00:00 bash 0 S   501 21516 21328  0  80   0 -  1434 wait   pts/2    00:00:00 python 0 Z   501 21517 21516  0  80   0 -     0 exit   pts/2    00:00:00 ls <defunct> 0 R   501 21518 21516  0  80   0 -   608 -      pts/2    00:00:00 ps after wait F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD 0 S   501 21328 21326  0  80   0 -  1574 wait   pts/2    00:00:00 bash 0 S   501 21516 21328  0  80   0 -  1467 wait   pts/2    00:00:00 python 0 R   501 21519 21516  0  80   0 -   608 -      pts/2    00:00:00 ps 

Otherwise, you can keep all the children in a list, and now and then .poll for their return codes. After every iteration, remember to remove from the list the children with return codes different than None (i.e. the finished ones).

Answer 2

Not using Popen.communicate() or call() will result in a zombie process.

If you don't need the output of the command, you can use subprocess.call():

>>> import subprocess >>> subprocess.call(['grep', 'jdoe', '/etc/passwd']) 0 

If the output is important, you should use Popen() and communicate() to get the stdout and stderr.

>>> from subprocess import Popen, PIPE >>> process = Popen(['ls', '-l', '/tmp'], stdout=PIPE, stderr=PIPE) >>> stdout, stderr = process.communicate() >>> stderr '' >>> print stdout total 0 -rw-r--r-- 1 jdoe jdoe 0 2010-05-03 17:05 bar -rw-r--r-- 1 jdoe jdoe 0 2010-05-03 17:05 baz -rw-r--r-- 1 jdoe jdoe 0 2010-05-03 17:05 foo