argparse subcommand error message

Question

Consider the following Python 2 code:

from argparse import ArgumentParser

p = ArgumentParser(prog="test")
p.add_argument('--bar')
sp = p.add_subparsers()
sp1 = sp.add_parser('foo')
sp1.add_argument('--baz')

p.parse_args(['foo', '--bar'])

The error message I get is:

usage: test [-h] [--bar BAR] {foo} ...
test: error: unrecognized arguments: --bar

This seems to imply that --bar is an unrecognized argument for test. But in fact it is an unrecognized argument for the foo subcommand.

I think the error message should be:

usage: test foo [-h] [--baz BAZ]
foo: error: unrecognized arguments: --bar

Is this a bug in argparse? Can I get configure argparse to give the correct error message?

Answer 1

If I tweak your script

p = ArgumentParser(prog="test")
p.add_argument('--bar')
sp = p.add_subparsers(dest='cmd')
sp1 = sp.add_parser('foo')
sp1.add_argument('--baz')
print p.parse_known_args()

the output is

1517:~/mypy$ python2.7 stack25333847.py foo --bar
(Namespace(bar=None, baz=None, cmd='foo'), ['--bar'])

Parser p encounters the foo, one of the allowed sp choices. So it now delegates parsing to the subparser, sp1. sp1 does not recognize --bar, so it returns that to the main parser in its list of unrecognized arguments. The default action is for the main parser to pass it on out, as though it(self) did not recognized the string.

Due its position after foo, --bar is not recognized by either parser. Same would go for ['foo', '--boo'].

Delegation to the subparser is done in the __call__ method of sp (the subparsers action). In part it reads:

def __call__(self, parser, namespace, values, option_string=None):
    ...
    # parse all the remaining options into the namespace
    # store any unrecognized options on the object, so that the top
    # level parser can decide what to do with them
    namespace, arg_strings = parser.parse_known_args(arg_strings, namespace)
    if arg_strings:
        vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
        getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)

So the handling of unrecognized_args is left, by design, to the main parser (the one who calls parse_args rather than parse_known_args).

---

A different error, such as omitting a value for --baz does generate the error message in the subparser:

1523:~/mypy$ python2.7 stack25333847.py foo --baz
usage: test foo [-h] [--baz BAZ]
test foo: error: argument --baz: expected one argument

---

I have figured out a way of generating:

usage: test foo [-h] [--baz BAZ]
test foo: error: unrecognized arguments: --bar

though it's not short and sweet. I subclass argparse._SubParsersAction; give it a new __call__ that uses parse_args instead of parse_known_args. I also have to change the main parser registry. (I can add the code if wanted).

Answer 2

I wouldn't necessarily say it's a bug, but rather a simplistic approach at generating an error message.

The default error message simply states "{PROG}: error: unrecognized arguments: {ALL_UNRECOGNIZED_ARGS}", regardless of what (sub-)parser the unrecognized arguments are associated with.

If you look at how parse_args() generates the error, it's pretty obvious that this information isn't readily available:

def parse_args(self, args=None, namespace=None):
    args, argv = self.parse_known_args(args, namespace)
    if argv:
        msg = _('unrecognized arguments: %s')
        self.error(msg % ' '.join(argv))
    return args

Calling args, argv = parse_known_args(args) will simply parse the known arguments and return them as the namespace args, and return all remaining, unknown arguments as argv.

So you could directly use parse_known_args() yourself. You could also store the subcommand name to the namespace using

sp = p.add_subparsers(dest='cmd')

and then access it using args.cmd to determine which subcommand the user tried to invoke.

Based on that, and depending on how simple your parser hierarchy is, you could maybe generate a more helpful error message (see parser.error()).

But it's not going to be perfect. If the user were to specify an unknown argument for both the main parser and a subparser, those two unrecognized arguments would remain in argv, and I see know obvious way of determining which of them was given to which command on the command line.