Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to get argparse to read arguments from a file with an option rather than prefix

I would like to know how to use python's argparse module to read arguments both from the command line and possibly from text files. I know of argparse's fromfile_prefix_chars but that's not exactly what I want. I want the behavior, but I don't want the syntax. I want an interface that looks like this:

$ python myprogram.py --foo 1 -A somefile.txt --bar 2

When argparse sees -A, it should stop reading from sys.argv or whatever I give it, and call a function I write that will read somefile.text and return a list of arguments. When the file is exhausted it should resume parsing sys.argv or whatever. It's important that the processing of the arguments in the file happen in order (ie: -foo should be processed, then the arguments in the file, then -bar, so that the arguments in the file may override --foo, and --bar might override what's in the file).

Is such a thing possible? Can I write a custom function that pushes new arguments onto argparse's stack, or something to that effect?

like image 503
Bryan Oakley Avatar asked Dec 11 '14 22:12

Bryan Oakley


People also ask

How do you make Argparse argument optional?

To add an optional argument, simply omit the required parameter in add_argument() . args = parser. parse_args()if args.

What does Argparse ArgumentParser () do?

>>> parser = argparse. ArgumentParser(description='Process some integers. ') The ArgumentParser object will hold all the information necessary to parse the command line into Python data types.

What does Nargs do in Argparse?

Number of Arguments If you want your parameters to accept a list of items you can specify nargs=n for how many arguments to accept. Note, if you set nargs=1 , it will return as a list not a single value.

What is dest in argument parser?

dest is equal to the first argument supplied to the add_argument() function, as illustrated. The second argument, radius_circle , is optional. A long option string --radius supplied to the add_argument() function is used as dest , as illustrated.


2 Answers

You can solve this by using a custom argparse.Action that opens the file, parses the file contents and then adds the arguments then.

For example this would be a very simple action:

class LoadFromFile (argparse.Action):
    def __call__ (self, parser, namespace, values, option_string = None):
        with values as f:
            # parse arguments in the file and store them in the target namespace
            parser.parse_args(f.read().split(), namespace)

Which you can the use like this:

parser = argparse.ArgumentParser()
# other arguments
parser.add_argument('--file', type=open, action=LoadFromFile)
args = parser.parse_args()

The resulting namespace in args will then also contain any configuration that was also loaded from the file where the file contained arguments in the same syntax as on the command line (e.g. --foo 1 --bar 2).

If you need a more sophisticated parsing, you can also parse the in-file configuration separately first and then selectively choose which values should be taken over. For example, since the arguments are evalutated in the order they are specified, it might make sense to prevent the configurations in the file from overwriting values that have been explicitly specified ont the command line. This would allow using the configuration file for defaults:

def __call__ (self, parser, namespace, values, option_string=None):
    with values as f:
        contents = f.read()

    # parse arguments in the file and store them in a blank namespace
    data = parser.parse_args(contents.split(), namespace=None)
    for k, v in vars(data).items():
        # set arguments in the target namespace if they haven’t been set yet
        if getattr(namespace, k, None) is not None:
            setattr(namespace, k, v)

Of course, you could also make the file reading a bit more complicated, for example read from JSON first.

like image 191
poke Avatar answered Oct 18 '22 08:10

poke


You commented that

I need to be able to write my own function to read that file and return the arguments (it's not in a one-argument-per-line format) –

There is a provision in the existing prefix-file handler to change how the file is read. The file is read by a 'private' method, parser._read_args_from_files, but it calls a simple public method that converts a line to strings, default one-argument-per-line action:

def convert_arg_line_to_args(self, arg_line):
    return [arg_line]

It was written this way so you could easily customize it. https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.convert_arg_line_to_args

A useful override of this method is one that treats each space-separated word as an argument:

def convert_arg_line_to_args(self, arg_line):
    for arg in arg_line.split():
        if not arg.strip():
            continue
        yield arg

In the test_argparse.py unittesting file there is a test case for this alternative.


But if you still want to trigger this read with an argument option, instead of a prefix character, then the custom Action approach is a good one.

You could though write your own function that processes argv before it is passed to the parser. It could be modeled on parser._read_args_from_files.

So you could write a function like:

def read_my_file(argv):
    # if there is a '-A' string in argv, replace it, and the following filename
    # with the contents of the file (as strings)
    # you can adapt code from _read_args_from_files
    new_argv = []
    for a in argv:
        ....
        # details left to user
    return new_argv

Then invoke your parser with:

parser.parse_args(read_my_file(sys.argv[1:]))

And yes, this could be wrapped in a ArgumentParser subclass.

like image 23
hpaulj Avatar answered Oct 18 '22 10:10

hpaulj