Using argparse with function that takes **kwargs argument

Question

I'm using argparse to take input and pass it to a function that takes as arguments two variables and **kwargs.

Here's my function:

import requests
import sys
import argparse


def location_by_coordinate(LAT, LNG, **kwargs):
    if not kwargs:
        coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
        r = requests.get(coordinate_url).text
    else:
        coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
        for key, value in kwargs.iteritems():
            if 'DISTANCE' in kwargs:
                distance = kwargs.get('DISTANCE')
                if distance > 5000:
                    print distance
                    print "max distance is 5000m, value is reassigned to default of 1000m"
                    distance = 1000
                    coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
                    r = requests.get(coordinate_url).text
                else:
                    pass
                    coordinate_url = "https://api.instagram.com/v1/locations/search?lat=%s&lng=%s&access_token=%s" % (LAT, LNG, current_token)
                    r = requests.get(coordinate_url).text
            if 'FACEBOOK_PLACES_ID' in kwargs:
                fb_places_id = kwargs.get('FACEBOOK_PLACES_ID')
                payload = {'FACEBOOK_PLACES_ID': '%s' % (fb_places_id), 'DISTANCE': '%s' % (DISTANCE)}
                r = requests.get(coordinate_url, params=payload).text
            if 'FOURSQUARE_ID' in kwargs:
                foursquare_id = kwargs.get('FOURSQUARE_ID')
                payload = {'FOURSQUARE_ID': '%s' % (foursquare_id), 'DISTANCE': '%s' % (DISTANCE)}
                r = requests.get(coordinate_url, params=payload).text
            if 'FOURSQUARE_V2_ID' in kwargs:
                foursquare_v2_id = kwargs.get('FOURSQUARE_V2_ID')
                payload = {'FOURSQUARE_V2_ID': '%s' % (foursquare_v2_id), 'DISTANCE': '%s' % (DISTANCE)}
                r = requests.get(coordinate_url, params=payload).text
    #print r
    return r

Given this function and its use of **kwargs, how should I setup the subparsers?

Here's how I've setup the command line parser thus far:

 def main():
        parser = argparse.ArgumentParser(description="API Endpoints tester")
        subparsers = parser.add_subparsers(dest="command", help="Available commands")

        location_by_parser = subparsers.add_parser("location_by_coordinate", help="location function")
        location_by_parser.add_argument("LAT", help="latitude")
        location_by_parser.add_argument("LNG", help="longitude")

        arguments = parser.parse_args(sys.argv[1:])
        arguments = vars(arguments)
        command = arguments.pop("command")
        if command == "location_by_coordinate":
            LAT, LNG = location_by_coordinate(**arguments)
        else:
            print "No command provided..."

    if __name__ == "__main__":
        main()

Obviously, the above main() function works fine with the location_by_coordinate() function when I call it at the command line like this:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148

But with the code the way it is currently, if I try:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148 DISTANCE=3000

Obviously, I get:

argstest.py: error: unrecognized arguments: DISTANCE=3000

But I'm not sure how to setup a subparser for **kwargs. If I try to setup a subparser like this:

location_by_parser.add_argument("**kwargs", help="**kwargs")

and then try that command again:

$ python argstest.py location_by_coordinate 40.5949799 -73.9495148 DISTANCE=3000

That doesn't work because the arguments object (which is a dictionary), becomes this:

{'LAT': '40.5949799', 'LNG': '-73.9495148', 'command': 'location_by_coordinate', '**kwargs': 'DISTANCE=3000'}

And this Traceback is returned:

Traceback (most recent call last):
  File "argstest.py", line 118, in <module>
    main()
  File "argstest.py", line 108, in main
    foo = location_by_coordinate(**arguments)
  File "argstest.py", line 40, in location_by_coordinate
    return r
UnboundLocalError: local variable 'r' referenced before assignment

How can I enable argparse to handle/to parse what is entered at the command line that is intended to be passed to the function via **kwargs?

Answer 1

Do you understand what is going on with the

{'LAT': '40.5949799', 'LNG': '-73.9495148', 'command': 'location_by_coordinate', '**kwargs': 'DISTANCE=3000'}

arguments dictionary? You defined a 'positional' argument with the name ('dest') of '**kwargs'. You could just as well named it 'foobar'. The parser assigned the string 'DISTANCE=3000' to that attribute in the args namespace, which turned into a dictionary key:value pair in arguments.

You could, of course, look for arguments['**kwargs'], and parse the value for yourself:

v = arguments['**kwargs']  # or pop if you prefer
if v is not None:
    k, v = v.split('=')
    arguments[k] = int(v)

It could be generalized to handle multiple pairs (defined with `nargs='*').

---

argparse does not handle arguments the same way as Python functions, so there's nothing exactly analogous the **kwargs.

The normal way to accept something like distance is with 'optionals' or flagged arguments.

parser.add_argument('-d','--distance', type=int, help=...)

which will accept

python argstest.py location_by_coordinate 40.5949799 -73.9495148 --distance=3000
python argstest.py location_by_coordinate 40.5949799 -73.9495148 --distance 3000
python argstest.py location_by_coordinate 40.5949799 -73.9495148 --d3000
python argstest.py location_by_coordinate 40.5949799 -73.9495148

It could also be setup to use --DISTANCE or other names. In the last case args namespace will have a default value for distance. The default default is None.

That's the straight forward way of adding kwarg like arguments to argparse.

Accepting arbitrary dictionary like pairs, distance:3000, distance=3000, has been asked before on SO. The answers have always been some variation of the parsing that I sketched above. It could be done in a custom Action class, or post parsing as I suggest.

oops, this answer is nearly a clone of one I wrote a few days ago: https://stackoverflow.com/a/33639147/901925

A similar 2011 question: Using argparse to parse arguments of form "arg= val"

Python argparse dict arg

=================================

(edit)

Example with a function that takes *args:

In [2]: import argparse
In [3]: def foo(*args, **kwargs):
   ...:     print('args',args)
   ...:     print('kwargs',kwargs)
   ...:     
In [4]: parser=argparse.ArgumentParser()
In [5]: parser.add_argument('arg1')
In [6]: parser.add_argument('arg2',nargs='+')

In [7]: args=parser.parse_args('one two three'.split())
In [8]: args
Out[8]: Namespace(arg1='one', arg2=['two', 'three'])

So I have 2 positional arguments, one with a single string value, the other with a list (due to the + nargs).

Call foo with these args attributes:

In [10]: foo(args.arg1)
args ('one',)
kwargs {}

In [11]: foo(args.arg1, args.arg2)
args ('one', ['two', 'three'])
kwargs {}

In [12]: foo(args.arg1, arg2=args.arg2)
args ('one',)
kwargs {'arg2': ['two', 'three']}

I defined 'positionals', but it would have worked just as well with 'optionals'. The distinction between positionals and optionals disappears in the namespace.

If I convert the namespace to a dictionary, I can pass values to foo in various ways, either through the *args or through **kwargs. It's all in how I call foo, not in how they appear in args or arguments. None of this is unique to argparse.

In [13]: arguments = vars(args)
In [14]: arguments
Out[14]: {'arg2': ['two', 'three'], 'arg1': 'one'}

In [15]: foo(arguments['arg2'], arguments['arg1'])
args (['two', 'three'], 'one')
kwargs {}

In [16]: foo(arguments['arg2'], arguments)
args (['two', 'three'], {'arg2': ['two', 'three'], 'arg1': 'one'})
kwargs {}

In [17]: foo(arguments['arg2'], **arguments)
args (['two', 'three'],)
kwargs {'arg2': ['two', 'three'], 'arg1': 'one'}

In [24]: foo(*arguments, **arguments)
args ('arg2', 'arg1')             # *args is the keys of arguments
kwargs {'arg2': ['two', 'three'], 'arg1': 'one'}

In [25]: foo(*arguments.values(), **arguments)
args (['two', 'three'], 'one')    # *args is the values of arguments
kwargs {'arg2': ['two', 'three'], 'arg1': 'one'}