I'm having problems redirecting stdio of another program using subprocess module. Just reading from stdout results in hanging, and Popen.communicate() works but it closes pipes after reading/writing. What's the easiest way to implement this?
I was playing around with this on windows:
import subprocess
proc = subprocess.Popen('python -c "while True: print \'Hi %s!\' % raw_input()"',
shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
while True:
proc.stdin.write('world\n')
proc_read = proc.stdout.readline()
if proc_read:
print proc_read
The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several older modules and functions: os. system os.
To start a new process, or in other words, a new subprocess in Python, you need to use the Popen function call. It is possible to pass two parameters in the function call. The first parameter is the program you want to start, and the second is the file argument.
communicate() writes input (there is no input in this case so it just closes subprocess' stdin to indicate to the subprocess that there is no more input), reads all output, and waits for the subprocess to exit.
Doesn't fit 100% to your example but helps to understand the underlying issue: Process P starts child C. Child C writes something to its stdout. stdout of C is a pipe which has a 4096 character buffer and the output is shorter than that. Now, C waits for some input. For C, everything is fine.
P waits for the output which will never come because the OS sees no reason to flush the output buffer of C (with so little data in it). Since P never gets the output of C, it will never write anything to C, so C hangs waiting for the input from P.
Fix: Use flush after every write to a pipe forcing the OS to send the data now.
In your case, adding proc.stdin.flush()
in the main while loop and a sys.stdout.flush()
in the child loop after the print should fix your problem.
You should also consider moving the code which reads from the other process into a thread. The idea here is that you can never know when the data will arrive and using a thread helps you to understand these issues while you write the code which processes the results.
At this place, I wanted to show you the new Python 2.6 documentation but it doesn't explain the flush issue, either :( Oh well ...
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With