Python subprocess: how to use pipes thrice? [duplicate]

Question

I'd like to use subprocess on the following line:

convert ../loxie-orig.png bmp:- | mkbitmap -f 2 -s 2 -t 0.48 | potrace -t 5 --progress -s -o ../DSC00232.svg

I found thank to other posts the subprocess documentation but in the example we use only twice pipe.

So, I try for two of the three commands and it works

p1 = subprocess.Popen(['convert', fileIn, 'bmp:-'], stdout=subprocess.PIPE)
# p2 = subprocess.Popen(['mkbitmap', '-f', '2', '-s', '2', '-t', '0.48'], stdout=subprocess.PIPE)
p3 = subprocess.Popen(['potrace', '-t' , '5', '-s' , '-o', fileOut], stdin=p1.stdout,stdout=subprocess.PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p3 exits.
output = p3.communicate()[0]

Can you help me for the third command?

Thank you very much.

Answer 1

Just add a third command following the same example:

p1 = subprocess.Popen(['convert', fileIn, 'bmp:-'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['mkbitmap', '-f', '2', '-s', '2', '-t', '0.48'], 
     stdin=p1.stdout, stdout=subprocess.PIPE)
p1.stdout.close()
p3 = subprocess.Popen(['potrace', '-t' , '5', '-s' , '-o', fileOut],        
     stdin=p2.stdout,stdout=subprocess.PIPE)
p2.stdout.close()

output = p3.communicate()[0]

Answer 2

Use subprocess.Popen() with the option shell=True, and you can pass it your entire command as a single string.

This is the simplest solution and makes it possible to embed a complicated pipeline in python without head-scratching; but in some cases it might not work, e.g. (as @torek commented) if there are spaces in the filenames passed for input or output. In that case, take the trouble to build up the robust solution in the accepted answer.

Answer 3

def runPipe(cmds):
try: 
    p1 = subprocess.Popen(cmds[0].split(' '), stdin = None, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
    prev = p1
    for cmd in cmds[1:]:
        p = subprocess.Popen(cmd.split(' '), stdin = prev.stdout, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
        prev = p
    stdout, stderr = p.communicate()
    p.wait()
    returncode = p.returncode
except Exception, e:
    stderr = str(e)
    returncode = -1
if returncode == 0:
    return (True, stdout.strip().split('\n'))
else:
    return (False, stderr)

Then execute it like:

runPipe(['ls -1','head -n 2', 'head -n 1'])