How to run a process with timeout and still get stdout at runtime

Question

The need:

1. Timeout after X seconds, and kill the process (and all the processes it opened) if timeout reached before the process ends gracefully.
2. Read ongoing output at runtime.
3. Work with processes that produce output, ones that don't, and ones that produce output, and then stop producing it (e.g. get stuck).
4. Run on Windows.
5. Run on Python 3.5.2.

Python 3 subprocess module has timeout built in, and I've also tried and implemented timeout myself using timer and using threads, but it doesn't work with the output. readline() is blocking or not? readlines() is definitely waiting for the process to end before spitting out all the output, which is not what I need (I need ongoing).

I am close to switching to node.js :-(

Answer 1

I would use asyncio for this kind of task.

Read IO from the process like in this accepted anwser: How to stream stdout/stderr from a child process using asyncio, and obtain its exit code after?

(I don't want to fully copy it here)

Wrap it in a timeout:

async def killer(trans, timeout):
    await asyncio.sleep(timeout)
    trans.kill()
    print ('killed!!')

trans, *other_stuff = loop.run_until_complete(
                           loop.subprocess_exec(
                                SubprocessProtocol, 'py', '-3', '-c', 'import time; time.sleep(6); print("Yay!")' ,
                                        ) 
                       )

asyncio.ensure_future(killer(trans, 5)) # 5 seconds timeout for the kill
loop.run_forever()

Have fun ...