Argparse - do not catch positional arguments with `nargs`.

Question

I am trying to write a function wo which you can parse a variable amount of arguments via argparse - I know I can do this via nargs="+". Sadly, the way argparse help works (and the way people generally write arguments in the CLI) puts the positional arguments last. This leads to my positional argument being caught as part of the optional arguments.

#!/usr/bin/python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("positional", help="my positional arg", type=int)
parser.add_argument("-o", "--optional", help="my optional arg", nargs='+', type=float)
args = parser.parse_args()
print args.positional, args.optional

running this as ./test.py -h shows the following usage instruction:

usage: test.py [-h] [-o OPTIONAL [OPTIONAL ...]] positional

but if I run ./test.py -o 0.21 0.11 0.33 0.13 100 gives me

test.py: error: too few arguments

to get a correct parsing of args, I have to run ./test.py 100 -o 0.21 0.11 0.33 0.13

So how do I:

• make argparse reformat the usage output so that it is less misleading, OR, even better:

• tell argparse to not catch the last element for the optional argument -o if it is the last in the list

?

Answer 1

There is a bug report on this: http://bugs.python.org/issue9338

argparse optionals with nargs='?', '*' or '+' can't be followed by positionals

A simple (user) fix is to use -- to separate postionals from optionals:

./test.py -o 0.21 0.11 0.33 0.13 -- 100

I wrote a patch that reserves some of the arguments for use by the positional. But it isn't a trivial one.

As for changing the usage line - the simplest thing is to write your own, e.g.:

usage: test.py [-h] positional [-o OPTIONAL [OPTIONAL ...]]
usage: test.py [-h] [-o OPTIONAL [OPTIONAL ...]] -- positional

I wouldn't recommend adding logic to the usage formatter to make this sort of change. I think it would get too complex.

Another quick fix is to turn this positional into an (required) optional. It gives the user complete freedom regarding their order, and might reduce confusion. If you don't want to confusion of a 'required optional' just give it a logical default.

usage: test.py [-h] [-o OPTIONAL [OPTIONAL ...]] -p POSITIONAL
usage: test.py [-h] [-o OPTIONAL [OPTIONAL ...]] [-p POS_WITH_DEFAULT]

---

One easy change to the Help_Formatter is to simply list the arguments in the order that they are defined. The normal way of modifying formatter behavior is to subclass it, and change one or two methods. Most of these methods are 'private' (_ prefix), so you do so with the realization that future code might change (slowly).

In this method, actions is the list of arguments, in the order in which they were defined. The default behavior is to split 'optionals' from 'positionals', and reassemble the list with positionals at the end. There's additional code that handles long lines that need wrapping. Normally it puts positionals on a separate line. I've omitted that.

class Formatter(argparse.HelpFormatter):
    # use defined argument order to display usage
    def _format_usage(self, usage, actions, groups, prefix):
        if prefix is None:
            prefix = 'usage: '

        # if usage is specified, use that
        if usage is not None:
            usage = usage % dict(prog=self._prog)

        # if no optionals or positionals are available, usage is just prog
        elif usage is None and not actions:
            usage = '%(prog)s' % dict(prog=self._prog)
        elif usage is None:
            prog = '%(prog)s' % dict(prog=self._prog)
            # build full usage string
            action_usage = self._format_actions_usage(actions, groups) # NEW
            usage = ' '.join([s for s in [prog, action_usage] if s])
            # omit the long line wrapping code
        # prefix with 'usage:'
        return '%s%s\n\n' % (prefix, usage)

parser = argparse.ArgumentParser(formatter_class=Formatter) 

Which produces a usage line like:

usage: stack26985650.py [-h] positional [-o OPTIONAL [OPTIONAL ...]]

Answer 2

Instead of using nargs="+", consider using action="append". This requires passing -o in front of each number, but it will not consume arguments unless you actually want it to.