How do I close the stdout-pipe when killing a process started with python subprocess Popen?

Question

I wonder if it is possible to shut down the communication pipe when killing a subprocess started in a different thread. If I do not call communicate() then kill() will work as expected, terminating the process after one second instead of five.

I found a discussion of a similar problem here, but I got no real answers. I assume that I either have to be able to close the pipe or to explicitly kill the sub-subprocess (which is "sleep" in the example) and kill that to unblock the pipe.

I also tried to find the answer her on SO, but I only found this and this and this, which do not directly address this problem as far as I can tell (?).

So the thing I want to do is to be able to run a command in a second thread and get all its output, but be able to kill it instantly when I so desire. I could go via a file and tail that or similar, but I think there should be a better way to do this?

import subprocess, time
from threading import Thread

process = None

def executeCommand(command, runCommand):
    Thread(target=runCommand, args=(command,)).start()

def runCommand(command):
    global process
    args = command.strip().split()
    process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE)

    for line in process.communicate():
        if line:
            print "process:", line,

if __name__ == '__main__':
    executeCommand("./ascript.sh", runCommand)
    time.sleep(1)
    process.kill()

This is the script:

#!/bin/bash
echo "sleeping five"
sleep 5
echo "slept five"

Output

$ time python poc.py 
process: sleeping five

real    0m5.053s
user    0m0.044s
sys 0m0.000s

Answer 1

I think the problem is that process.kill() only kills the immediate child process (bash), not the sub-processes of the bash script.

The problem and solution are described here:

• http://www.doughellmann.com/PyMOTW/subprocess/#process-groups-sessions
• How to terminate a python subprocess launched with shell=True

Use Popen(..., preexec_fn=os.setsid) to create a process group and os.pgkill to kill the entire process group. eg

import os
import signal
import subprocess
import time
from threading import Thread

process = None

def executeCommand(command, runCommand):
    Thread(target=runCommand, args=(command,)).start()

def runCommand(command):
    global process
    args = command.strip().split()
    process = subprocess.Popen(
        args, shell=False, stdout=subprocess.PIPE, preexec_fn=os.setsid)

    for line in process.communicate():
        if line:
            print "process:", line,

if __name__ == '__main__':
    executeCommand("./ascript.sh", runCommand)
    time.sleep(1)
    os.killpg(process.pid, signal.SIGKILL)

---

$ time python poc.py 
process: sleeping five

real    0m1.051s
user    0m0.032s
sys 0m0.020s