What difference between subprocess.call() and subprocess.Popen() makes PIPE less secure for the former?

Question

I've had a look at the documentation for both of them.

This question is prompted by J.F.'s comment here: Retrieving the output of subprocess.call()

The current Python documentation for subprocess.call() says the following about using PIPE for subprocess.call():

Note Do not use stdout=PIPE or stderr=PIPE with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

Python 2.7 subprocess.call():

Note Do not use stdout=PIPE or stderr=PIPE with this function as that can deadlock based on the child process output volume. Use Popen with the communicate() method when you need pipes.

Python 2.6 includes no such warnings.

Also, the subprocess.call() and subprocess.check_call() don't seem to have a way to access their output, except for using stdout=PIPE with communicate():

https://docs.python.org/2.6/library/subprocess.html#convenience-functions

Note that if you want to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE. Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

https://docs.python.org/2.6/library/subprocess.html#subprocess.Popen.communicate

What difference between subprocess.call() and subprocess.Popen() makes PIPE less secure for subprocess.call()?

More Specific: Why does subprocess.call() "deadlock based on the child process output volume.", and not Popen()?

Answer 1

call() is just Popen().wait() (± error handling).

You should not use stdout=PIPE with call() because it does not read from the pipe and therefore the child process will hang as soon as it fills the corresponding OS pipe buffer. Here's a picture that shows how data flows in command1 | command2 shell pipeline:

It does not matter what your Python version is -- the pipe buffer (look at the picture) is outside of your Python process. Python 3 does not use C stdio but it affects only the internal buffering. When the internal buffer is flushed the data goes into the pipe. If command2 (your parent Python program) does not read from the pipe then command1 (the child process e.g., started by call()) will hang as soon as the pipe buffer is full (pipe_size = fcntl(p.stdout, F_GETPIPE_SZ) ~65K on my Linux box (max value is /proc/sys/fs/pipe-max-size ~1M)).

You may use stdout=PIPE if you read from the pipe later e.g., using Popen.communicate() method. You could also read from process.stdout (the file object that represents the pipe) directly.