Dynamic arguments for Python's argparse

Question

I have a need in Python to create a list of arguments dynamically. I've created a script to demonstrate this, named args.py, shown below:

#!/usr/bin/python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-args_file', default = 'args.txt')

with open(parser.parse_args().args_file, 'r') as f:
    args = f.readlines()

for arg in args:
    parser.add_argument('-' + arg.strip())

dynamic_args = parser.parse_args()

print dynamic_args

I also have a text file in the same folder, named args.txt, also shown below:

arg1
arg2
arg3

As expected, running args.py with no arguments results in:

Namespace(arg1=None, arg2=None, arg3=None, args_file='args.txt')

However, what I'm having trouble with is running my script with the -h argument. I would like the help to display the arguments found in the args_file, as seen in the example below:

usage: args.py [-h] [-args_file ARGS_FILE] [-arg1 ARG1] [-arg2 ARG2]
               [-arg3 ARG3]

What I'm seeing instead is:

usage: args.py [-h] [-args_file ARGS_FILE]

Moreover, if I run the script interactively (i.e. python -i arg.py), and at the interactive prompt type the command "parser.print_usage()", I get the wanted response (showing the -argN arguments). Also, typing "arg.py -arg1 1" or "arg.py arg1 1" result in "unrecognized arguments".

I've tried everything I can think of, but I've been unsuccessful thus far. Do any of the Python aficionados have any suggestions?

Answer 1

From Hazen's answer, using parents option makes it easier.

https://docs.python.org/3/library/argparse.html#parents

import argparse
argfile_parser = argparse.ArgumentParser(add_help=False)
argfile_parser.add_argument('-args_file', default = 'args.txt')
full_parser = argparse.ArgumentParser(parents=[argfile_parser])
with open(argfile_parser.parse_known_args()[0].args_file, 'r') as f:
    for arg in f:
        full_parser.add_argument('-' + arg.strip())

dynamic_args = full_parser.parse_args()

print dynamic_args

Answer 2

As Martjin pointed out, you can omit the help from the parser the first time. The other thing to do is to use parse_known_args the first time, so you only parse the args_file.

In cases like this, I like to keep things clear by using a throwaway parser for the first parse, and a full parser for the final parse:

import argparse
argfile_parser = argparse.ArgumentParser(add_help=False)
full_parser = argparse.ArgumentParser()
argfile_parser.add_argument('-args_file', default = 'args.txt')
full_parser.add_argument('-args_file', default = 'args.txt')
with open(argfile_parser.parse_known_args()[0].args_file, 'r') as f:
    for arg in f:
        full_parser.add_argument('-' + arg.strip())

dynamic_args = full_parser.parse_args()

print dynamic_args

For testing, I added a file args2.txt:

argA
argB
argC

And I think the result is what you're looking for:

lap:~$ python tmp.py -h
usage: tmp.py [-h] [-args_file ARGS_FILE] [-arg1 ARG1] [-arg2 ARG2]
              [-arg3 ARG3]

optional arguments:
  -h, --help            show this help message and exit
  -args_file ARGS_FILE
  -arg1 ARG1
  -arg2 ARG2
  -arg3 ARG3
lap:~$ python tmp.py -args_file args2.txt
Namespace(argA=None, argB=None, argC=None, args_file='args2.txt')
lap:~$ python tmp.py -h -args_file args2.txt
usage: tmp.py [-h] [-args_file ARGS_FILE] [-argA ARGA] [-argB ARGB]
              [-argC ARGC]

optional arguments:
  -h, --help            show this help message and exit
  -args_file ARGS_FILE
  -argA ARGA
  -argB ARGB
  -argC ARGC
lap:~$ python tmp.py -arg1 foo
Namespace(arg1='foo', arg2=None, arg3=None, args_file='args.txt')
lap:~$ python tmp.py -args_file args2.txt -argA bar
Namespace(argA='bar', argB=None, argC=None, args_file='args2.txt')