Python subprocess interaction, why does my process work with Popen.communicate, but not Popen.stdout.read()?

Question

I am trying to communicate with a command-line chat bot with Python using the subprocess module. (http://howie.sourceforge.net/ using the compiled win32 binary, I have my reasons!)

This works:

proc = Popen('Howie/howie.exe', stdout=PIPE,stderr=STDOUT,stdin=PIPE)
output = proc.communicate()

But Popen.communicate waits for the process to terminate (and sends it EOF?), I want to be able to interact with it. The apparent solution for this was to read stdout / write stdin like so:

This doesn't work:

proc = Popen('Howie/howie.exe', stdout=PIPE,stderr=STDOUT,stdin=PIPE)
while True: print proc.stdout.readline()

(Note that I am actually using more complex code based on http://code.activestate.com/recipes/440554/ but the issue is the same.)

The problem is, the second approach works perfectly for communicating to cmd, but when I run the chatbot, nothing. So my question is, how is this different in capturing output to using Popen.communicate()?

i.e. I can use the second approach to use the command line as per normal, until I run the chatbot, at which point I stop receiving output. Using the first approach correctly displays the first few lines of output from the bot, but leaves me unable to interact with it.

Answer 1

One major difference between the two is that communicate() closes stdin after sending the data. I don't know about your particular case, but in many cases this means that if a process is awaiting the end of the user input, he will get it when communicate() is used, and will never get it when the code blocks on read() or readline().

Try adding Popen.stdin.close() first and see if it affects your case.

Answer 2

If you want to interact with the program after sending the EOF, rather than using Popen.stdin.close(), you can manually send the command-line End Of File character, which has the same effect but leaves stdin open.

In Python this character's escape sequence is '\x1a'.