How can I define global options with sub-parsers in python argparse?

Question

I'm trying to figure out how to add global option in a sub-parser scenario with pythons arparse library.

Right now my code looks like this:

def parseArgs(self):
    parent_parser = argparse.ArgumentParser(add_help=False)
    parent_parser.add_argument('--debug', default=False, required=False,
        action='store_true', dest="debug", help='debug flag')

    main_parser = argparse.ArgumentParser()
    main_parser.add_argument('--debug', default=False, required=False,
        action='store_true', dest="debug", help='debug flag')

    service_subparsers = main_parser.add_subparsers(title="category",
        dest="category")
    agent_parser = service_subparsers.add_parser("agent",
        help="agent commands", parents=[parent_parser])
    return main_parser.parse_args()

This works for the command line ./test --help and the --debug option is listed as global:

usage: test [-h] [--debug] {agent} ...

optional arguments:
  -h, --help  show this help message and exit
  --debug     debug flag

category:
  {agent}
    agent     agent commands

However when I trigger the agent sub-parser with the command line ./test agent --help the --debug option is now no longer listed as a global option but as an option for the sub-parser. Also it must now specified as ./test agent --debug and ./test --debug agent no longer works:

usage: test agent [-h] [--debug]

optional arguments:
  -h, --help  show this help message and exit
  --debug     debug flag

What I'd like to be able to do is define --debug is global so that it can always be specified for all sub-parsers and appropriately listed as such in the help output.

Answer 1

main_parser fills in the defaults into namespace (False for debug); if it encounters --debug it changes debug to True. When it sees the agent string, it calls the subparser, passing it the remaining argument strings, and the namespace that it has been using.

Now the subparser does the normal parser things - if fills in the defaults for its arguments, setting default to False. If it encounters --debug in the remaining strings, it changes that to True. Otherwise it leaves it as is. Once it is done, it passes the namespace back to the main parser, which then returns it to your code.

So for

myprog.py --debug agent --debug

namespace(debug=False) has flipped from False to True to False and back to True.

This a consequence of sharing the same dest for both the main parser (I don't like the use of 'global' in this context), and the subparser.

There was a bug/issue that tried to change the behavior a bit, passing the subparser a 'virgin' namespace, and then somehow merging its result with the main one. But that produced some backward compatibility issues. I could look it up if needed.

For now, trying to define the same optional in both the main and subparser is bound to create confusion for you and your user.

If I change the parent to

parent_parser.add_argument('--Debug', action='store_true', help='debug flag')

(no need for default, or the dest if it is sames as the option flag)

the resulting namespace will look like

1721:~/mypy$ python stack37933480.py --debug agent --Debug
Namespace(Debug=True, category='agent', debug=True)

Or I could define

parent_parser.add_argument('--debug', dest='debug1', action='store_true', help='debug flag')

and get:

1724:~/mypy$ python stack37933480.py --debug agent --debug
Namespace(category='agent', debug=True, debug1=True)

Same flag in both places, but different entries in the namespace. After parsing I could do something like:

args.debug = args.debug or args.debug1

to unify the two flags. Your user will see '--debug' regardless of which help asks for.

Sorry if the description is a bit long winded, but I think it's important to understand the behavior first. Then solutions become more apparent.

In this case the use of a parent doesn't complicate the issue. I assume you are using it just to add this debug to all subparsers.

Another option is to just define debug for the main parser. Yes, it will be missing from the subparsers help, but you can always add a note in the description.

===================

The subparser definition takes a prog parameter. If not given it is defined base on the main prog.

If I add prog as:

agent_parser = service_subparsers.add_parser("agent",
    prog='myprog.py [--debug] agent',
    help="agent commands", parents=[parent_parser])

subparser usage becomes:

1824:~/mypy$ python3 stack37933480.py agent -h
usage: myprog.py [--debug] agent [-h] [--debug]

or I can add that prog to the add_subparsers definition

service_subparsers = main_parser.add_subparsers(title="category",
    prog='myprog.py [--debug]',
    dest="category")

Check the code for that method to see how it constructs the default usage prefix. It includes main positionals, but not optionals.

http://bugs.python.org/issue9351 - in this patch the original developer thought that users would expect the subparser definition of an argument should override the main parser's values and actions. You were, on the other hand, expecting the main definition to have priority.

http://bugs.python.org/issue24251 - but the correction proposed in 9351 caused problems for other users. That's why I think it is better not to define the same dest in the main and sub. It is hard to satisfy everyone's expectations.