Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to elegantly parse argumens in python before expensive imports?

I have a script, which parses a few arguments, and has some expensive imports, but those imports are only needed if the user gives valid input arguments, otherwise the program exits. Also, when the user says python script.py --help, there is no need for those expensive imports to be executed at all.

I can think of such a script:

import argparse

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--argument', type=str)
    args = parser.parse_args()
    return args

if __name__ == "__main__":
    args = parse_args()

import gensim # expensive import
import blahblahblah

def the_rest_of_the_code(args):
    pass

if __name__ == "__main__":
    the_rest_of_the_code(args)

This does the job, but it doesn't look elegant to me. Any better suggestions for the task?

EDIT: the import is really expensive:

$ time python -c "import gensim"
Using TensorFlow backend.

real    0m12.257s
user    0m10.756s
sys 0m0.348s
like image 239
adrin Avatar asked Nov 10 '17 09:11

adrin


People also ask

How do you parse an argument in Python?

To parse options, you first create an ArgumentParser instance and add declarations for the options you want to support it using the add_argument() method. In each add_argument() call, the dest argument specifies the name of an attribute where the result of parsing will be placed.

What is Store_true in Python?

The store_true option automatically creates a default value of False. Likewise, store_false will default to True when the command-line argument is not present. The source for this behavior is succinct and clear: http://hg.python.org/cpython/file/2.7/Lib/argparse.py#l861.

What does parse_args return?

parse_args() returns two values: options, an object containing values for all of your options— e.g. if "--file" takes a single string argument, then options. file will be the filename supplied by the user, or None if the user did not supply that option.

What does Nargs do in Argparse?

By default, argparse will look for a single argument, shown above in the filename example. If you want your parameters to accept a list of items you can specify nargs=n for how many arguments to accept. Note, if you set nargs=1 , it will return as a list not a single value.


2 Answers

You can import conditionally, or in a try block, or just about anywhere in code.

So you could do something like this:

import cheaplib

if __name__ == "__main__":
    args = parse_args()
    if expensive_arg in args:
        import expensivelib
    do_stuff(args)

Or even more clearly, only import the lib in the function that will use it.

def expensive_function():
    import expensivelib
    ...
like image 175
pills Avatar answered Sep 19 '22 12:09

pills


Not sure it's better than what you already have, but you can load it lazily:

def load_gensim():
    global gensim
    import gensim

If you only want to make sure the arguments make sense, you can have a wrapper main module that checks the arguments and then loads another module and call it.

main.py:

args = check_args()
if args is not None:
    import mymodule
    mymodule.main(args)

mymodule.py:

import gensim
def main(args):
    # do work
like image 36
Elazar Avatar answered Sep 21 '22 12:09

Elazar