I'd like to run a system process, intercept the output, and modify it real-time, line by line, in a Python script.
My best attempt, which waits for the process to complete before printing, is:
#!/usr/bin/env python
import subprocess
cmd = "waitsome.py"
proc = subprocess.Popen(cmd, shell=True, bufsize=256, stdout=subprocess.PIPE)
for line in proc.stdout:
print ">>> " + line.rstrip()
The script waitsome.py
simply prints a line every half a second:
#!/usr/bin/env python
import time
from sys import stdout
print "Starting"
for i in range(0,20):
time.sleep(0.5)
print "Hello, iteration", i
stdout.flush()
Is there an easy solution to get subprocess
to allow iterating over the output in real time? Do I have to use threads?
Once upon a time, I scripted in Perl, and this was a piece of cake:
open(CMD, "waitsome.py |");
while (<CMD>) {
print ">>> $_";
}
close(CMD);
Looping over a file unavoidably buffers things in pretty large chunks -- a known issue with all Python 2.* implementations. It works as you intend in Python 3.1, with the final loop being slightly different:
for line in proc.stdout:
print(">>> " + str(line.rstrip()))
If upgrading to Python 3.1 is impractical (and I know it will often be!), go the other way and write the loop in an old-fashioned manner -- the following version of the loop does work as you intend in Python 2.*:
while True:
line = proc.stdout.readline()
if not line:
break
print ">>> " + line.rstrip()
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With