Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When should I use `wait` instead of `communicate` in subprocess?

In the document of wait (http://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait), it says:

Warning

This will deadlock when using stdout=PIPE and/or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use communicate() to avoid that.

From this, I think communicate could replace all usage of wait() if retcode is not need. And even when the stdout or stdin are not PIPE, I can also replace wait() by communicate().

Is that right? Thanks!

like image 208
Hanfei Sun Avatar asked Nov 23 '12 02:11

Hanfei Sun


People also ask

Does subprocess call wait for completion?

The subprocess module provides a function named call. This function allows you to call another program, wait for the command to complete and then return the return code.

What is difference between subprocess Popen and call?

Popen is more general than subprocess. call . Popen doesn't block, allowing you to interact with the process while it's running, or continue with other things in your Python program. The call to Popen returns a Popen object.

What does Popen communicate do?

In subprocess , Popen() can interact with the three channels and redirect each stream to an external file, or to a special value called PIPE . An additional method, called communicate() , is used to read from the stdout and write on the stdin .

Do you need to close Popen?

Popen do we need to close the connection or subprocess automatically closes the connection? Usually, the examples in the official documentation are complete. There the connection is not closed. So you do not need to close most probably.


1 Answers

I suspect (the docs don't explicitly state it as of 2.6) in the case where you don't use PIPEs communicate() is reduced to wait(). So if you don't use PIPEs it should be OK to replace wait().

In the case where you do use PIPEs you can overflow memory buffer (see communicate() note) just as you can fill up OS pipe buffer, so either one is not going to work if you're dealing with a lot of output.

On a practical note I had communicate (at least in 2.4) give me one character per line from programs whose output is line-based, that wasn't useful to put it mildly.

Also, what do you mean by "retcode is not needed"? -- I believe it sets Popen.returncode just as wait() does.

like image 160
dima Avatar answered Oct 14 '22 20:10

dima