Killing a script launched in a Process via os.system()

Question

I have a python script which launches several processes. Each process basically just calls a shell script:

from multiprocessing import Process
import os
import logging

def thread_method(n = 4):
    global logger
    command = "~/Scripts/run.sh " + str(n) + " >> /var/log/mylog.log"
    if (debug): logger.debug(command)
    os.system(command)

I launch several of these threads, which are meant to run in the background. I want to have a timeout on these threads, such that if it exceeds the timeout, they are killed:

t = []
for x in range(10):
    try:
        t.append(Process(target=thread_method, args=(x,) ) )
        t[-1].start()
    except Exception as e:
        logger.error("Error: unable to start thread")
        logger.error("Error message: " + str(e))
logger.info("Waiting up to 60 seconds to allow threads to finish")
t[0].join(60)
for n in range(len(t)):
    if t[n].is_alive():
    logger.info(str(n) + " is still alive after 60 seconds, forcibly terminating")
     t[n].terminate()

The problem is that calling terminate() on the process threads isn't killing the launched run.sh script - it continues running in the background until I either force kill it from the command line, or it finishes internally. Is there a way to have terminate also kill the subshell created by os.system()?

Answer 1

You should use an event to signal the worker to terminate, run the subprocess with subprocess module, then terminate it with Popen.terminate(). Calling Process.terminate() will not allow it worker to clean up. See the documentation for Process.terminate().

Answer 2

Use subprocess instead, whose objects have a terminate() method explicitly for this.

Answer 3

In Python 3.3, the subprocess module supports a timeout: http://docs.python.org/dev/library/subprocess.html

For other solutions regarding Python 2.x, please have a look in this thread: Using module 'subprocess' with timeout