Using python to run other programs

Question

I have a command that works great on the command line. It has lots of arguments like cmd --thing foo --stuff bar -a b input output

I want to run this from python and block waiting for it to complete. As the script prints things to stdout and stderr I want it to be immediately shown to the user.

What is the right module for this?

I've tried:

---

import commands
output = commands.getoutput("cmd --thing foo --stuff bar -a b input output")
print output

this works great except the stdout isn't returned until the end.

---

import os
os.system("cmd --thing foo --stuff bar -a b input output")

this prints all the output when the cmd is actually finished.

---

import subprocess
subprocess.call(["cmd", "--thing foo", "--stuff bar", "-a b", "input", "output"])

this doesn't pass the parameters correctly somehow (I haven't been able to find the exact problem, but cmd is rejecting my input). If I put echo as the first parameter, it prints out the command which works perfectly when I paste it directly into the terminal.

---

import subprocess
subprocess.call("cmd --thing foo --stuff bar -a b input output")

exactly the same as above.

Answer 1

You have to quote each field separately, ie. split the options from their arguments.

import subprocess
output = subprocess.call(["cmd", "--thing", "foo", "--stuff", "bar", "-a", "b", "input", "output"])

otherwise you are effectively running cmd like this

$ cmd --thing\ foo --stuff\ bar -a\ b input output

To get the output into a pipe you need to call it slightly differently

import subprocess
output = subprocess.Popen(["cmd", "--thing", "foo", "--stuff", "bar", "-a", "b", "input", "output"],stdout=subprocess.PIPE)
output.stdout   #  <open file '<fdopen>', mode 'rb'>