Wildcard not working in subprocess call using shlex

Question

Language: Python v2.6.2

OS: AIX 5.3

I'm using Python to restore some files from a backup to a test system - all commands are called in the manner below, however some just plain don't want to work.

#!/usr/bin/python
import subprocess, shlex

cmd = 'sudo rm -rf /work/TEST/*'
arg = shlex.split(cmd)

# This does not work
p = subprocess.Popen(arg)

# This, however, works just fine
p = subprocess.Popen(cmd, shell=True)

If I remove the *'s from the commands they work fine (well, they work as they should without the wildcards, which is unfortauntely not what I want).

I really do not want to use shell=True for obvious security reasons, however there are a couple of other commands that basically do the same thing. If there is a wildcard in the command it just won't work - it executes without error, just doesn't do anything.

Interestingly the following command (parsed through shlex):

sudo mv /work/testrestore/production/* /work/TESTC

Produces the following:

mv: 0653-401 Cannot rename /work/testrestore/production/* to /work/TESTC/*: A file or directory in the path name does not exist.

It's as if unix is now trying to move a file named * rather then using * as a wildcard. Is this typical behaviour of shlex?

Edit: I have tried escaping the * with a \, also tried changing from single quotes to double.. not that I expected that to do anything.

Answer 1

For replacing the * with what it means, you either need the shell or you need the glob module. So the easiest way would be shell=True (if the command is constant, I do not see any security holes).

Another approach would be

#!/usr/bin/python
import subprocess
import shlex
import glob

cmd = 'sudo rm -rf /work/TEST/*'
arg = shlex.split(cmd)
arg = arg[:-1] + glob.glob(arg[-1])

# This should work now
p = subprocess.Popen(arg)

or, if you would nevertheless append the path by yourself,

cmd = 'sudo rm -rf'
basearg = shlex.split(cmd)
arg = basearg + glob.glob(path+"/*")