Escaping for proper shell injection prevention

Question

I need to run some shell commands from a Lua interpreter embedded into another Mac/Windows application where shell commands are the only way to achieve certain things, like opening a help page in a browser. If I have a list of arguments (which might be the result of user input), how can I escape each argument to prevent trouble?

Inspired by this article an easy solution seems to be to escape all non-alphanumeric characters, on Unix-like systems with \, on Windows with ^. As far as I can tell, this prevents that any argument will cause

• execution of another command because of on intervening newline, ; (Unix) or & (Windows)
• command substitution on Unix with $ or `
• variable evaluation on Windows with %
• redirection with <, | and >

In addition, any character that on the respective platform works as escape character will be escaped properly.

This seems sound to me, but are there any pitfalls I might have missed? I know that in bash, \ followed by a newline will effectively remove the newline, which is not a problem here.

EDIT

My conclusion: There is no single mechanism that by swapping the escape character will work on both Windows and *nix. It turns out it is not so straightforward to make sure that a Windows program actually sees the command line arguments we want it to see as splitting the command string into arguments on Windows is not handled by the shell but by the called program itself.

Therefore two layers of escaping need to be taken into account:

1. First, the Windows shell will process what we give it. What it might do is variable substitution at %, splitting into multiple commands at & or piping to another command at |.
2. Then, it will hand on a single command string to the called program which the program will split, ideally but not necessarily following the rules described by Microsoft.

Assuming it follows these rules, one can work one's way backwards, first escaping to these rules, then escaping further for the shell.

Answer 1

Calling sub-processes with dynamic arguments is prone to error and danger, and many languages don't provide good mechanisms to protect the developer. In Python, for example, os.system() is no longer recommended, instead the subprocess module provides a proper mechanism for safely making system calls. In particular, you pass subprocess.run() a list of arguments, rather than a single string, thereby avoiding needing to implement any error-prone escaping in the first place.

A quick search for a subprocess-like tool for Lua uncovered lua-subprocess, which doesn't appear to be being actively developed, but might still be better than trying to implement proper escaping yourself.

If you must do so, take a look at the Python code for shlex.quote() (source) - it properly escapes an input string for use "in a shell command line":

# use single quotes, and put single quotes into double quotes
# the string $'b is then quoted as '$'"'"'b'
return "'" + s.replace("'", "'\"'\"'") + "'"

You ought to be able to replicate that in Lua.