Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unbuffered read from process using subprocess in Python

Tags:

I am trying to read from a process that produces long and time-consuming output. However, I want to catch it's output as and when it is produced. But using something like the following seems to be buffering the command's output, so I end up getting the output lines all at once:

p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, bufsize=0)     for line in p.stdout:         print line 

I am trying this on MacOS 10.5

like image 635
Abhi Avatar asked Jul 26 '09 03:07

Abhi


People also ask

How do I get output to run from subprocess?

To capture the output of the subprocess. run method, use an additional argument named “capture_output=True”. You can individually access stdout and stderr values by using “output. stdout” and “output.

Is subprocess thread safe Python?

subprocess. Process class is not thread safe. The Concurrency and multithreading in asyncio section.

What is the difference between subprocess call and Popen?

Popen is more general than subprocess. call . Popen doesn't block, allowing you to interact with the process while it's running, or continue with other things in your Python program. The call to Popen returns a Popen object.

How does subprocess run get return value?

subprocess. check_output() is the one that runs the command and returns the return value. If you want the output write your value to STDOUT and use check_output() to get the value.


1 Answers

The file iterator is doing some internal buffering on its own. Try this:

line = p.stdout.readline() while line:     print line     line = p.stdout.readline() 

You also need to make sure the process you are running is actually flushing its output buffers frequently.

like image 128
brendan Avatar answered Nov 09 '22 07:11

brendan