Why does this command work with os.system() but not subprocess.Popen()?

Question

I want to delete a number of jobs from a q. The command to delete the job is qdel JOBid.

Initially, I tried to use the subprocess module, but I got an error: #!/usr/bin/env python

 import sys, os, subprocess as sp

 lo = sys.argv[1]
 hi = sys.argv[2]

 lo = int(lo)
 hi = int(hi)


for i in range(lo,hi):
    print "i is %d"%i
    p=sp.Popen(['qdel %d'%i],stdout=sp.PIPE)
    #os.system('qdel %d'%i)

So this did not work. The error I got was

Traceback (most recent call last):
File "del.py", line 14, in <module>
p=sp.Popen(['qdel %d'%i],stdout=sp.PIPE)
File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
errread, errwrite)
File "/usr/lib64/python2.6/subprocess.py", line 1228, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Then I commented out the subprocess line and used os and it worked immediately. I think I don't fully understand the subprocess module

#!/usr/bin/env python

import sys, os, subprocess as sp

lo = sys.argv[1]
hi = sys.argv[2]

lo = int(lo)
hi = int(hi)


    for i in range(lo,hi):
         print "i is %d"%i
         #p=sp.Popen(['qdel %d'%i],stdout=sp.PIPE)
         os.system('qdel %d'%i)

The above code worked flawlessly. I just want to know why and what the advantages are of the subprocess module. Also, I am using a unix shell

Answer 1

If you read manual, you can see that your call to Popen is wrong: you should pass not a single command, but an array of arguments:

p=sp.Popen(['qdel', '%d'%i],stdout=sp.PIPE)

Alternatively, as sc0tt's answer points out, you can use shell=True, but this has some disadvantages in more complex cases: you would have to manually escape all the variable data in the command in case it contains, for example, filenames with spaces or anything much more potentially harmful (like ;)

Answer 2

You want to use shell=True on your Popen call.

p=sp.Popen(['qdel %d'%i], shell=True, stdout=sp.PIPE)