How can I create an argparse mutually exclusive group with multiple positional parameters?

Question

I'm trying to parse command-line arguments such that the three possibilities below are possible:

script
script file1 file2 file3 …
script -p pattern

Thus, the list of files is optional. If a -p pattern option is specified, then nothing else can be on the command line. Said in a "usage" format, it would probably look like this:

script [-p pattern | file [file …]]

I thought the way to do this with Python's argparse module would be like this:

parser = argparse.ArgumentParser(prog=base)
group = parser.add_mutually_exclusive_group()
group.add_argument('-p', '--pattern', help="Operate on files that match the glob pattern")
group.add_argument('files', nargs="*", help="files to operate on")
args = parser.parse_args()

But Python complains that my positional argument needs to be optional:

Traceback (most recent call last):
  File "script", line 92, in <module>
    group.add_argument('files', nargs="*", help="files to operate on")
…
ValueError: mutually exclusive arguments must be optional

But the argparse documentation says that the "*" argument to nargs meant that it is optional.

I haven't been able to find any other value for nargs that does the trick either. The closest I've come is using nargs="?", but that only grabs one file, not an optional list of any number.

Is it possible to compose this kind of argument syntax using argparse?

Answer 1

short answer

Add a default to the * positional

long

The code that is raising the error is,

    if action.required:
        msg = _('mutually exclusive arguments must be optional')
        raise ValueError(msg)

If I add a * to the parser, I see that the required attribute is set:

In [396]: a=p.add_argument('bar',nargs='*')
In [397]: a
Out[397]: _StoreAction(option_strings=[], dest='bar', nargs='*', const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [398]: a.required
Out[398]: True

while for a ? it would be False. I'll have dig a bit further in the code to see why the difference. It could be a bug or overlooked 'feature', or there might a good reason. A tricky thing with 'optional' positionals is that no-answer is an answer, that is, an empty list of values is valid.

In [399]: args=p.parse_args([])
In [400]: args
Out[400]: Namespace(bar=[], ....)

So the mutually_exclusive has to have some way to distinguish between a default [] and real [].

For now I'd suggest using --files, a flagged argument rather than a positional one if you expect argparse to perform the mutually exclusive testing.

---

The code that sets the required attribute of a positional is:

    # mark positional arguments as required if at least one is
    # always required
    if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]:
        kwargs['required'] = True
    if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs:
        kwargs['required'] = True

So the solution is to specify a default for the *

In [401]: p=argparse.ArgumentParser()
In [402]: g=p.add_mutually_exclusive_group()
In [403]: g.add_argument('--foo')
Out[403]: _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [404]: g.add_argument('files',nargs='*',default=None)
Out[404]: _StoreAction(option_strings=[], dest='files', nargs='*', const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [405]: p.parse_args([])
Out[405]: Namespace(files=[], foo=None)

The default could even be []. The parser is able to distinguish between the default you provide and the one it uses if none is given.

oops - default=None was wrong. It passes the add_argument and required test, but produces the mutually_exclusive error. Details lie in how the code distinguishes between user defined defaults and the automatic ones. So use anything but None.

I don't see anything in the documentation about this. I'll have to check the bug/issues to see it the topic has been discussed. It's probably come up on SO before as well.