Communicate with subprocess without waiting for the subprocess to terminate on windows

Question

I have a simple echoprocess.py:

import sys

while True:
    data = sys.stdin.read()
    sys.stdout.write("Here is the data: " + str(data))

And a parentprocess.py

from subprocess import Popen, PIPE

proc = Popen(["C:/python27/python.exe", "echoprocess.py"],
             stdin = PIPE,
             sdtout = PIPE)

proc.stdin.write("hello")
print proc.stdout.read()

This just hangs until echoprocess.py is terminated. I want to communicate with this subprocess multiple times without having to restart it again. Is this kind of interprocess communication possible with the Python subprocess module on Windows?

Answer 1

The main problem is with the line...

print proc.stdout.read()

The read() method when used with no parameters will read all data until EOF, which will not occur until the subprocess terminates.

You'll probably be okay with line-by-line reading, so you can use...

proc.stdin.write("hello\n")
print proc.stdout.readline()

...otherwise you'll have to work out some others means of delimiting 'messages'.

You'll have to make a similar change to echoprocess.py, i.e. change...

data = sys.stdin.read()

...to...

data = sys.stdin.readline()

You may also have issues with output buffering, so it may be necessary to flush() the buffer after doing a write.

---

Putting all this together, if you change echoprocess.py to...

import sys

while True:
    data = sys.stdin.readline()
    sys.stdout.write("Here is the data: " + str(data))
    sys.stdout.flush()

...and parentprocess.py to...

from subprocess import Popen, PIPE

proc = Popen(["C:/python27/python.exe", "echoprocess.py"],
             stdin = PIPE,
             stdout = PIPE)

proc.stdin.write("hello\n")
proc.stdin.flush()
print proc.stdout.readline()

...it should work the way you expect it to.