python to wait for shell command to complete

Question

I am running script to unrar some files and the remove the rar files afterwards. I am doing this by running the command through shell. I have tried several different ways to make the script wait until it's done unpacking the files, but it still goes ahead and deletes the file before they are done being used.

I have tried the code below, which is a no go. I have tried to see if i could get the wait() to work, but also no luck.

Any ideas? running python 2.7

EDIT: I want the script to run the command :)

            p = subprocess.Popen('unrar e ' + root + '/' + i + ' ' + testfolder,
                                 bufsize=2048, shell=True,
                                 stdin=subprocess.PIPE)
            p.stdin.write('e')
            p.communicate()

for root, dirs, files in os.walk(testfolder):
    for i in files:

        print 'Deleting rar files'
        os.remove(i)

for i in os.listdir(testfolder):
    if os.path.isdir(testfolder + i):
        shutil.rmtree(testfolder + i)

Answer 1

This is evil:

p = subprocess.Popen('unrar e ' + root + '/' + i + ' ' + testfolder,
        bufsize=2048, shell=True, stdin=subprocess.PIPE)

Instead,

p = subprocess.Popen(['unrar', 'e', '%s/%s' % (root, i), testfolder],
        bufsize=2048, stdin=subprocess.PIPE)
p.stdin.write('e')
p.wait()
if p.returncode == 0:
    pass # put code that must only run if successful here.

By passing an exact array rather than a string to Popen and not using shell=True, a filename with a space in it can't be interpreted as a more than one arguments, or a subshell command, or some other potentially malicious thing (think of a file with $(rm -rf ..) in its name).

Then, after calling p.wait() (there's no need for p.communicate() when you aren't capturing stderr or stdout), you must check p.returncode to determine whether the process was successful, and only proceed on to delete files if p.returncode == 0 (indicating success).

Your initial diagnosis, that p.communicate() is returning while the unrar process is still running, is not feasible; p.communicate() and p.wait() do not work that way.

---

If running across ssh, this changes a bit:

import pipes # in Python 2.x; in 3.x, use shlex.quote() instead
p = subprocess.Popen(['ssh', ' '.join(
      [pipes.quote(s) for s in ['unrar', 'e', '%s/%s' % (root, i), testfolder]])