Why does `subprocess.check_call(..., stderr=sys.stdout)` fail in Python 2.6?

Question

Python 2.6.9 (unknown, Mar  7 2016, 11:15:18) 
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import subprocess
>>> subprocess.check_call(['echo', 'hi'], stderr=sys.stdout)
echo: write error: Bad file descriptor
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/subprocess.py", line 488, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['echo', 'hi']' returned non-zero exit status 1

This command, subprocess.check_call(['echo', 'hi'], stderr=sys.stdout), works just fine in Python 2.7 and Python 3. What is Python 2.6 doing differently?

Answer 1

The bug is discussed here:

Transcript to reproduce in Python 2.6.5:

>>> import subprocess, sys
>>> subprocess.call(('echo', 'foo'), stderr=sys.stdout)
echo: write: Bad file descriptor
1
>>> 

Expected behavior:

>>> import subprocess, sys
>>> subprocess.call(('echo', 'foo'), stderr=sys.stdout)
foo
0
>>> 

This happens because we've asked the child's stderr to be redirected, but not its stdout. So in _execute_child, errwrite is 1 while c2pwrite is None. So fd 1 (errwrite) correctly gets duped to 2. But then, since errwrite is not None and it's not in (p2cread, c2pwrite, 2), the child closes fd 1.

The equivalent thing happens if you supply stdout=sys.stderr and the child attempts to write to its stderr.

I've attached a patch to fix this. It simply adds 2 and 2 to the list of fds not to close for c2pwrite and errwrite, respectively.

This patch is against the 2.6.5 release.

There is also a workaround, in case anyone else is affected by this bug before a fix has been released:

>>> import os, subprocess, sys
>>> subprocess.call(('echo', 'foo'), stderr=os.dup(sys.stdout.fileno()))
foo
0
>>> 

It was fixed in 2.7 with this patch on a related issue.