Popen: differences between python 2 and 3

Question

I'm trying to port a wrapper for gnuplot from python2 to python3. Most errors are easy to fix but the communication with the project seems to behave unexpectedly. I've isolated the problem in the following (ugly) snippet.

cmd = ['gnuplot']

p = subprocess.Popen(cmd, stdin=subprocess.PIPE,
                     stdout=subprocess.PIPE,
                     stderr=subprocess.PIPE)

p.stdin.write("set terminal dumb 80 40\n")
p.stdin.write("plot '-' w p ls 1, '-' w p ls 2, '-' w p ls 3 \n")
p.stdin.write("1 2 3\n")
p.stdin.write("2 3 4\n")
p.stdin.write("\ne\n")
p.stdin.write("e\n")
p.stdin.write("e\n")
while True:
    print(p.stdout.read(1),end="")

This code works and produces and print a result in python2 but fails in python3. First it complains about bytes and string, so I add universal_newlines=True. From there I cannot understand why it outputs nothing on stdout and print this in stderr: line 4: warning: Skipping data file with no valid points line 5: warning: Skipping data file with no valid points

Clearly the problem is somewhere in the encoding or communication because the commands I'm issuing are identical but I don't know where to look or how to debug this.

Any suggestion is welcome.

Answer 1

Python 3 makes a stronger distinction between bytes and strings than Python 2. As such, you must encode the strings you are sending to standard input as bytes, and must decode the bytes you receive from standard output to strings. Also, when I tried your program, I had to add p.stdin.close() as Charles suggested so the program wouldn't hang while gnuplot was waiting for input.

Here's the working version of your code I came up with:

import subprocess

cmd = ['gnuplot']

p = subprocess.Popen(cmd, stdin=subprocess.PIPE,
                     stdout=subprocess.PIPE,
                     stderr=subprocess.PIPE)

p.stdin.write("set terminal dumb 80 40\n".encode())
p.stdin.write("plot '-' w p ls 1, '-' w p ls 2, '-' w p ls 3\n".encode())
p.stdin.write("1 2 3\n".encode())
p.stdin.write("2 3 4\n".encode())
p.stdin.write("\ne\n".encode())
p.stdin.write("e\n".encode())
p.stdin.write("e\n".encode())
p.stdin.close()

print(p.stdout.read().decode())
print(p.stderr.read().decode())