Python - subprocesses and the python shell

Question

I am trying to shell out to a non-python subprocess and allow it to inherit the stdin and stdout from python. - i am using subprocess.Popen

This would probably work if I am calling from a console, but it definitely doesn't work when I am using the python shell

(I am using IDLE by the way)

Is there any way to convince python to allow a non python subprocess to print it's stdout to the python shell?

Answer 1

This works both from a script and from the interactive interpreter, but not from IDLE:

subprocess.Popen(whatever, stdin=sys.stdout, stdout=sys.stdin)

You can't use the objects which IDLE assigns to sys.stdin and sys.stdout as arguments to subprocess.Popen. These objects (the interfaces to the IDLE shell window) are file-like, but they're not real file handles with fileno attributes, and Unix-like operating systems require a fileno to be specified as the stdin or stdout for a subprocess. I cannot speak for Windows, but I imagine it has similar requirements.

Answer 2

Taymon's answer addresses your question directly in that IDLE's stdin/stdout are actually file-like objects and not the standard file streams associated with a console/terminal. Moreover, in Windows IDLE runs with pythonw.exe, which doesn't even have an attached win32 console.

That said, if you just need the output from a program to be printed to the user in real time, then in many cases (but not all) you can read the output line by line and echo it accordingly. The following works for me in Windows IDLE. It demonstrates reading from a piped stdout line by line. It also shows what happens if the process buffers the pipe, in which case readline will block until either the buffer is full or the pipe closes. This buffering can be manually disabled with some programs (such as the Python interpreter's -u option), and there are workarounds for Unix such as stdbuf.

test1.py

import sys
import subprocess

def test(cmd):
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, 
                         stderr=subprocess.PIPE)
    it = iter(p.stdout.readline, b'')
    for line in it:
        print(line.rstrip().decode('ascii'))

print('Testing buffered subprocess...')
test([sys.executable, 'test2.py'])

print('\nTesting unbuffered subprocess...')
#-u: unbuffered binary stdout and stderr
test([sys.executable, '-u', 'test2.py']) 

test2.py:

import time

for i in range(5):
    print(i)
    time.sleep(1)

The output in IDLE should be the following, with the first set of digits printed all at once after a delay and the second set printed line by line.

Testing buffered subprocess...
0
1
2
3
4

Testing unbuffered subprocess...
0
1
2
3
4