Is it possible to use argparse to capture an arbitrary set of optional arguments?

Question

Is it possible to use argparse to capture an arbitrary set of optional arguments?

For example both the following should be accepted as inputs:

python script.py required_arg1 --var1 value1 --var2 value2 --var3 value3  python script.py required_arg1 --varA valueA --var2 value2 --varB valueB 

a priori I don't know what optional arguments would be specified receive but would handle them accordingly.

Answer 1

This is kind of a hackish way, but it works well:

Check, which arguments are not added and add them

import argparse parser = argparse.ArgumentParser() parser.add_argument("foo") parser.add_argument("-bar", type=int) # parser can have any arguments, whatever you want!  parsed, unknown = parser.parse_known_args() # this is an 'internal' method # which returns 'parsed', the same as what parse_args() would return # and 'unknown', the remainder of that # the difference to parse_args() is that it does not exit when it finds redundant arguments  for arg in unknown:     if arg.startswith(("-", "--")):         # you can pass any arguments to add_argument         parser.add_argument(arg.split('=')[0], type=<your type>, ...)  args = parser.parse_args() 

For example:

python3 arbitrary_parser.py ha -bar 12 -extra1 value1 -extra2 value2 

Then the result would be

args = Namespace(bar=12, foo='ha', extra1='value1' extra2='value2') 

Answer 2

Possible? possibly, but I wouldn't recommend it. argparse is the not best tool for parsing this kind of input, or conversely, this a poor argument specification from an argparse perspective.

Have you thought about what the usage line should look like? How would explain this to your users?

How would you parse this working from sys.argv directly? It looks like you could collect 3 pieces:

 prog = sys.argv[0]  arg1 = sys.argv[1]  keys = sys.argv[2::2]  # maybe strip -- off each  values = sys.argv[3::2]  kvdict = {k:v for k, v in zip(keys, values)} 

There are other SO questions asking about generic key:value pairs. Things like:

 --args key1:value1 key2:value2 

This can be handled with nargs='*' and an action that splits each input string on : (or =) and stores things by key.

Your requirement is least amenable to argparse use because it requires bypassing the whole idea of matching argument flags with strings in argv. It requires, some how, turning off all the normal argparse parsing.

Looks like I suggested the same thing a couple of years ago

Parse non-pre-defined argument

or earlier

Using argparse to parse arguments of form "arg= val"