Can't get argparse to read quoted string with dashes in it?

Question

Is there a way to make argparse recognize anything between two quotes as a single argument? It seems to keep seeing the dashes and assuming that it's the start of a new option

I have something like:

mainparser = argparse.ArgumentParser() subparsers = mainparser.add_subparsers(dest='subcommand') parser = subparsers.add_parser('queue') parser.add_argument('-env', '--extraEnvVars', type=str,                         help='String of extra arguments to be passed to model.') ...other arguments added to parser... 

But when I run:

python Application.py queue -env "-s WHATEVER -e COOL STUFF" 

It gives me:

Application.py queue: error: argument -env/--extraEnvVars: expected one argument 

If I leave off the first dash, it works totally fine, but it's kind of crucial that I be able to pass in a string with dashes in it. I've tried escaping it with \ , which causes it to succeed but adds the \ to the argument string Does anyone know how to get around this? This happens whether or not -s is an argument in parser.

EDIT: I'm using Python 2.7.

EDIT2:

python Application.py -env " -env" 

works perfectly fine, but

python Application.py -env "-env" 

does not.

EDIT3: Looks like this is actually a bug that's being debated already: http://www.gossamer-threads.com/lists/python/bugs/89529, http://python.6.x6.nabble.com/issue9334-argparse-does-not-accept-options-taking-arguments-beginning-with-dash-regression-from-optp-td578790.html. It's only in 2.7 and not in optparse.

EDIT4: The current open bug report is: http://bugs.python.org/issue9334

Answer 1

Updated answer:

You can put an equals sign when you call it:

python Application.py -env="-env" 

---

Original answer:

I too have had troubles doing what you are trying to do, but there is a workaround build into argparse, which is the parse_known_args method. This will let all arguments that you haven't defined pass through the parser with the assumption that you would use them for a subprocess. The drawbacks are that you won't get error reporting with bad arguments, and you will have to make sure that there is no collision between your options and your subprocess's options.

Another option could be to force the user's to use a plus instead of a minus:

python Application.py -e "+s WHATEVER +e COOL STUFF" 

and then you change the '+' to '-' in post processing before passing to your subprocess.

Answer 2

This issue is discussed in depth in http://bugs.python.org/issue9334. Most of the activity was in 2011. I added a patch last year, but there's quite a backlog of argparse patches.

At issue is the potential ambiguity in a string like '--env', or "-s WHATEVER -e COOL STUFF" when it follows an option that takes an argument.

optparse does a simple left to right parse. The first --env is an option flag that takes one argument, so it consumes the next, regardless of what it looks like. argparse, on the other hand, loops through the strings twice. First it categorizes them as 'O' or 'A' (option flag or argument). On the second loop it consumes them, using a re like pattern matching to handle variable nargs values. In this case it looks like we have OO, two flags and no arguments.

The solution when using argparse is to make sure an argument string will not be confused for an option flag. Possibilities that have been shown here (and in the bug issue) include:

--env="--env"  # clearly defines the argument.  --env " --env"  # other non - character --env "--env "  # space after  --env "--env one two"  # but not '--env "-env one two"' 

By itself '--env' looks like a flag (even when quoted, see sys.argv), but when followed by other strings it does not. But "-env one two" has problems because it can be parsed as ['-e','nv one two'], a `'-e' flag followed by a string (or even more options).

-- and nargs=argparse.PARSER can also be used to force argparse to view all following strings as arguments. But they only work at the end of argument lists.

There is a proposed patch in issue9334 to add a args_default_to_positional=True mode. In this mode, the parser only classifies strings as option flags if it can clearly match them with defined arguments. Thus '--one' in '--env --one' would be classed as as an argument. But the second '--env' in '--env --env' would still be classed as an option flag.

---

Expanding on the related case in

Using argparse with argument values that begin with a dash ("-")

parser = argparse.ArgumentParser(prog="PROG") parser.add_argument("-f", "--force", default=False, action="store_true") parser.add_argument("-e", "--extra") args = parser.parse_args() print(args) 

produces

1513:~/mypy$ python3 stack16174992.py --extra "--foo one" Namespace(extra='--foo one', force=False) 1513:~/mypy$ python3 stack16174992.py --extra "-foo one" usage: PROG [-h] [-f] [-e EXTRA] PROG: error: argument -e/--extra: expected one argument 1513:~/mypy$ python3 stack16174992.py --extra "-bar one" Namespace(extra='-bar one', force=False) 1514:~/mypy$ python3 stack16174992.py -fe one Namespace(extra='one', force=True) 

The "-foo one" case fails because the -foo is interpreted as the -f flag plus unspecified extras. This is the same action that allows -fe to be interpreted as ['-f','-e'].

If I change the nargs to REMAINDER (not PARSER), everything after -e is interpreted as arguments for that flag:

parser.add_argument("-e", "--extra", nargs=argparse.REMAINDER) 

All cases work. Note the value is a list. And quotes are not needed:

1518:~/mypy$ python3 stack16174992.py --extra "--foo one" Namespace(extra=['--foo one'], force=False) 1519:~/mypy$ python3 stack16174992.py --extra "-foo one" Namespace(extra=['-foo one'], force=False) 1519:~/mypy$ python3 stack16174992.py --extra "-bar one" Namespace(extra=['-bar one'], force=False) 1519:~/mypy$ python3 stack16174992.py -fe one Namespace(extra=['one'], force=True) 1520:~/mypy$ python3 stack16174992.py --extra --foo one Namespace(extra=['--foo', 'one'], force=False) 1521:~/mypy$ python3 stack16174992.py --extra -foo one Namespace(extra=['-foo', 'one'], force=False) 

argparse.REMAINDER is like '*', except it takes everything that follows, whether it looks like a flag or not. argparse.PARSER is more like '+', in that it expects a positional like argument first. It's the nargs that subparsers uses.

This uses of REMAINDER is documented, https://docs.python.org/3/library/argparse.html#nargs