Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Understanding subprocess.TimeoutExpired, want to kill a child process after a timeout occurs

can somebody help me understand how the timeout parameter works in the subprocess module, and how to properly use the subprocess.TimeoutExpired exception?

My usecase is I have a main runner program that runs a child process as part of its repeated routine. The child process is known to hang from time to time. I would like to prevent this hang from holding everything up.

I was thinking i could use the timeout parameter to give the child only so long to run. However, in my sample programs below the behavior is not what I expected. When parent.py is run, it does launch child.py and I can see the output of child.py counting up. After 4s parent.py does get the subprocess.TimeoutExpired exception, however output from child.py keeps coming. This leads me to believe that child.py process has not actually been killed. The docs seem to suggest it will be killed however:

The timeout argument is passed to Popen.wait(). If the timeout expires, the child process will be killed and then waited for again. The TimeoutExpired exception will be re-raised after the child process has terminated.

So how might I finish this problem off? Do I need to somehow kill my child process myself when I receive the timeout exception?

Thanks for the help.

parent.py

#!/usr/bin/env python3

import os
import sys
import subprocess

p = subprocess.Popen("/path/to/python3 /path/to/child.py", shell=True)
try:
    p.wait(timeout=4)
except subprocess.TimeoutExpired:
    print("we got a timeout. exiting")
    sys.exit(1)

child.py

#!/usr/bin/env python3

import os
import sys
import time

for i in range(200):
    print("i is {}".format(i))
    time.sleep(1)
like image 771
D.C. Avatar asked Jan 24 '15 23:01

D.C.


People also ask

How do you kill a process in python subprocess?

subprocess. Popen. kill(proc) is exactly the same as proc. kill() FYI.

Does Popen have a timeout?

The timeout argument is passed to Popen. communicate() . If the timeout expires, the child process will be killed and waited for. The TimeoutExpired exception will be re-raised after the child process has terminated.

Is subprocess a child process?

Many of these programs are easily executable as a command from the shell in Unix/Linux and command prompt on windows. Python provides a module named subprocess which lets us execute these programs by creating an independent child process.


2 Answers

It does appear you need to add a call:

p.terminate()

before the sys.exit in your parent process per the current documentation the bit you quote only applies to subprocess.call, and that is not what you are using here.

like image 118
Alex Martelli Avatar answered Sep 19 '22 09:09

Alex Martelli


Taken from the documentation:

If the process does not terminate after timeout seconds, raise a TimeoutExpired exception. It is safe to catch this exception and retry the wait.

Meaning that you can catch the exception and unless you kill the process (or python) the process will continue executing. In your case since you're doing a

sys.exit(1)

Then python itself will be terminated and the Popen object will be garbage collected.

like image 25
Greg Avatar answered Sep 21 '22 09:09

Greg