Is Python3 shlex.quote() safe?

Question

I execute some code in shell using

subprocess.Popen('echo '+user_string+' | pipe to some string manipulation tools',
    shell=True)

where user_string is from an untrusted source.

Is it safe enough to use shlex.quote() for escaping the input?

Answer 1

I'm necromancing this because it's the top Google hit for "is shlex.quote() safe", and while the accepted answer seems correct, there's a lot of pitfalls to point out.

shlex.quote() escapes the shell's parsing, but it does not escape the argument parser of the command you're calling, and some additional tool-specific escaping needs to be done manually, especially if your string starts with a dash (-).

Most (but not all) tools accept -- as an argument, and anything afterward is interpreted as verbatim. You can prepend "-- " if the string starts with "-". Example: rm -- --help removes the file called --help.

When dealing with file names, you can prepend "./" if the string starts with "-": rm ./--help.

In the case of your example with echo, neither escape is sufficient: When attempting to echo the string -e, echo -- -e gives the wrong result, you'll need something like echo -e \x2de. This demonstrates that there's no universal bulletproof way to escape program arguments.

The safest route is to bypass the shell by avoiding shell=True or os.system() if the string involves any user-supplied data.

In your case, set stdin=subprocess.PIPE and pass the user_string as an argument to communicate(). Then you can even leave your original invocation as-is with shell=True:

subprocess.Popen(
    'pipe to some string manipulation tools',
    shell=True,
    stdin=subprocess.PIPE
).communicate(user_input_string.encode())