Python Asyncio subprocess never finishes

Question

i have a simple python program that I'm using to test asyncio with subprocesses:

import sys, time
for x in range(100):
    print("processing (%s/100)    " % x)
    sys.stdout.flush()
print("enjoy")
sys.stdout.flush()

Running this on the command line produces the desired results.

However, when called from asyncio, it never finishes

process = yield from asyncio.create_subprocess_exec(
    *["python", "program.py"],
    stdout=async_subprocess.PIPE,
    stderr=async_subprocess.STDOUT,
    cwd=working_dir
)

# this never finishes
yield from process.communicate()

ps ax shows this process is <defunct>, not sure what that means

Answer 1

I suspect your issue is just related to how you're calling asyncio.create_subprocess_exec and process.communiate(). This complete example works fine for me:

import asyncio
from asyncio import subprocess

@asyncio.coroutine
def do_work():
    process = yield from asyncio.create_subprocess_exec(
        *["python", "program.py"],
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT
    )

    stdout, _= yield from process.communicate()
    print(stdout)

if __name__  == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(do_work())

You have to place code that uses yield from inside of a asyncio.coroutine, and then call it inside an event loop (using loop.run_until_complete), for it to behave the way you want it to.