Python: switching from optparse to argparse

Question

After switching from optparse to argparse - I'm experiencing strange errors. Argparse parse args only if leave no space:

myScript.py -oOpt

or put an equal sign:

myScript.py -o=Opt

and it doesn't work the normal way:

myScript.py -o Opt

Here's my argparse initialization:

#!/usr/bin/env python
# to get description use the -h flag

import argparse, os, sys


# ======================
# Python2.7 is expected:

if sys.version_info[0] != 2 or sys.version_info[1] < 7:
    sys.exit('This program needs Python2.7+')


# ==========
# preambule:

desc = """Enter dirs in the current dir and makes gro out of state.cpt there."""
# parser = argparse.ArgumentParser()
parser = argparse.ArgumentParser(description=desc, version='2.3', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('-w', '--workWith',
                    help = 'to specify a Gromacs exec suffix', 
                    dest = 'wW',
                    action = 'store',
                    default = '-4.5.5-single',
                    )
parser.add_argument('-g', '--gro',
                    help = '.gro postfix: <nameOfTheDir><postfix>.gro', 
                    dest = 'myGroPostfix',
                    action = 'store',
                    default = "_membrane",
                    )
parser.add_argument('-H', '--here',
                    help = 'toggles - single (current) dir behaviour (the output will be state.gro)', 
                    dest = 'Here',
                    action = 'store_true',
                    )
parser.add_argument('-D', '--dirs',
                    help = 'include these dirs (python\'s rgxp in SINGLE quotes), defaults to \'\'', 
                    dest = 'inclDirs',
                    action = 'store',
                    default = '',
                    )

args = parser.parse_args()


print args.wW

Edit:

Even more:

 gmx_bk-simulate-mems.py -j bk-runs-mpi.bash -p 1 -w="-4.5.5-double_non-parallel_gcc" 2&> ../`date +%Y-%b-%d-%H%M%S`.log &

gives:

 gmx_bk-simulate-mems.py: error: unrecognized arguments: 2

it looks like argparse treats 2&> as option (or 2&> and ../date +%Y-%b-%d-%H%M%S.log as options)!

Edit 2:

So to summarize:

• For argparse - "-4.5.5-double_non-parallel_gcc" is a bad option name - and that's why it is required to write is as -w="-4.5.5-double_non-parallel_gcc". For optparse and bash (!) this is fine. bash even gives an error at -w="-4.5.5-double_non-parallel_gcc" - it thinks that the arg is ="-4.5.5-double_non-parallel_gcc" (!);

• There's no such thing as 2&>. 2> should might be used and it gives no errors;

• This is shell who splits the line into args, not python;

• argparse is much better than optparse.

Answer 1

First, it is necessary to make a small distinction. The argparse module does not parse your command-line arguments, the shell does. The shell is responsible for transforming the line you type in the shell into tokens, which are then passed to sys.argv, a python array/sequence of command-line arguments. The argparse module simply makes sense of what appears in sys.argv.

This distinction will clarify both of the "errors" you noticed. First, consider -w "-4.5.5-double_non-parallel_gcc" (note the lack of equal sign). The shell parses these two tokens as -w and -4.5.5-double_non-parallel_gcc, and both of these strings are passed to sys.argv. Without the equal sign, this appears to be two options: -w (with no argument) and -4 with .5.5-double_non-parallel_gcc as an argument. You need the equals sign so that everything is parsed as a single token.

EDITED SECTION

As for 2&>, it is impossible for argparse to control whether a given token is treated as an argument or not. If something appears in sys.argv, that means your shell is treating it as an argument.

The telltale sign here is the error message. Note that the message is not unrecognized arguments: 2&>, but rather, unrecognized arguments: 2. Your shell recognizes "&>" as output redirection, and parses the rest of the line (including the log file) accordingly. The only argument being passed is the "2", because 2&> is not a real type of redirection. (&> already covers both stderr and stdout, so what would the 2 add to it?)

In a comment, you claimed that optparse could "handle" the "2&>". That is not actually the case. The optparse module did exactly what argparse does, but optparse does not validate positional arguments like argparse does. In fact, optparse is letting a real programming error (in this case, using 2&> as a type of shell redirection) slip by undetected! You should post your original optparse code, but I suspect you parsed your arguments as follows:

opt, args = parser.parse_args()

Your script takes no positional arguments, so I imagine you did nothing else with args afterwards. But if you were to inspect args, you would find that 2 was considered a positional argument!

In general, if a script takes no positional arguments and you use optparse, it is good practice to verify that you receive no positional arguments, like so:

opt, args = parser.parse_args()
if args:
    parser.error("script takes no positional arguments")

The argparse module does that work for you, which is what puts it miles ahead of optparse (among other reasons).