How to read the remaining of a command line with argparse?

Question

First I'm sorry for this awful title. The fact that I'm not able to correctly formulate the summary is probably 80% my current problem.

I am writing a command-line program to wrap other command-line programs, like this:

my_wrapper.py --some options -for --the wrapper original_program --and its -options

This command would replace in some cases the call for the original program:

original_program --and its -options

I would like to extract the options needed for the wrapper but keep the original program name and arguments as is, in the exact same order. I cannot make any assumption on the arguments of the wrapped program.

Is it possible to do that with argparse? And how?

I think with the (deprecated) optparse module it would be as simple as reading the args from the line

 (options, args) = parser.parse_args()

For the curious ones, the goal of the wrapper is to assist users for submitting jobs on an HPC cluster. Its duty will be to take care of all our special requirements for writing a custom script, launching it and retrieving results.

Answer 1

You may use -- as separator. Simple example:

import argparse


parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('commands', type=str, nargs='+')
print(parser.parse_args())

And then call like this:

$ ./prog.py -- somecommand --somearg --someotherarg
Namespace(commands=['somecommand', '--somearg', '--someotherarg'])

From bash manual:

A -- signals the end of options and disables further option processing. Any arguments after the -- are treated as filenames and arguments. An argument of - is equivalent to --.

Other options is using parse_known_args Example:

import argparse


parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('command', type=str)
parser.add_argument('-s', '--somearg', type=str)
args, unknown = parser.parse_known_args()
print(args)
print(unknown)

And run:

$ ./prog2.py --somearg option test --other
Namespace(command='test', somearg='option')
['--other']

$ ./prog2.py  --somearg option test --other --second
Namespace(command='test', somearg='option')
['--other', '--second']

As said @hpaulj you may use nargs=argparse.REMAINDER. Example from docs:

>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print parser.parse_args('--foo B cmd --arg1 XX ZZ'.split())
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')