Specifying keyword arguments with *args and **kws

Question

I've found a behavior in Python that has baffled and irritated me and I was wondering what I got wrong.

I have a function which should take an arbitrary number of arguments and keywords, but in addition should have some default-valued keywords that comprise it's actual interface:

def foo(my_keyword0 = None, my_keyword1 = 'default', *args, **kws):
    for argument in args:
        print argument

The problem is that if I try calling foo(1, 2, 3) I'll only get the printout for 3 and the values 1 and 2 will override my keyword arguments.

On the other hand if I try moving my keywords after the *args or after the **kws it will cause a syntax error. The only solution I found to the problem was to extract the keyword arguments from **kws and setting default values to them:

def foo(*args, **kws):
    my_keyword0 = None if 'my_keyword0' not in kws else kws.pop('my_keyword0')
    my_keyword0 = 'default' if 'my_keyword1' not in kws else kws.pop('my_keyword1')
    for argument in args:
        print argument

This is horrible both because it forces me to add pointless code and because the function signature becomes harder to understand - you have to actually read the functions code rather than just look at its interface.

What am I missing? Isn't there some better way to do this?

Answer 1

Function arguments with default values are still positional arguments, and thus the result you see is correct. When you specify a default value for a parameter, you are not creating a keyword argument. Default values are simply used when the parameters are not provided by a function call.

>>> def some_function(argument="Default"):
...     # argument can be set either using positional parameters or keywords
...     print argument
... 
>>> some_function()    # argument not provided -> uses default value
Default
>>> some_function(5)    # argument provided, uses it
5
>>> some_function(argument=5)    # equivalent to the above one
5
>>> def some_function(argument="Default", *args):
...     print (argument, args)
... 
>>> some_function()   #argument not provided, so uses the default and *args is empty
('Default', ())
>>> some_function(5)   # argument provided, and thus uses it. *args are empty
(5, ())
>>> some_function(5, 1, 2, 3)   # argument provided, and thus uses it. *args not empty
(5, (1, 2, 3))
>>> some_function(1, 2, 3, argument=5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: some_function() got multiple values for keyword argument 'argument'

Note the last error message: as you can see the 1 got assigned to argument, and then python discovered the keyword referring to argument again, and thus raised an error. The *args are assigned only after assigning all possible positional arguments.

In python2 there is no way to define a keyword-only value other than using **kwargs. As a workaround you could do something like:

def my_function(a,b,c,d,*args, **kwargs):
    default_dict = {
        'my_keyword1': TheDefaultValue,
        'my_keyword2': TheDefaultValue2,
    }
    default_dict.update(kwargs)    #overwrite defaults if user provided them
    if not (set(default_dict) <= set('all', 'the', 'possible', 'keywords')):
        # if you want to do error checking on kwargs, even though at that
        # point why use kwargs at all?
        raise TypeError('Invalid keywords')
    keyword1 = default_dict['keyword1']
    # etc.

In python3 you can define keyword-only arguments:

def my_function(a,b,c,*args, keyword, only=True): pass
    # ... 

Note that keyword-only does not imply that it should have a default value.

Answer 2

In Python, non-keyword arguments cannot appear after a keyword argument, so the function signature you're trying to use is impossible:

foo(my_keyword0="baa", my_keyword1="boo", 1, 2, 3, bar="baz", spam="eggs")  # won't work

If you think about this, there are very valid reasons for this restriction (hint: keyword arguments may be presented in any order and positional arguments are... well, positional)

From the documentation:

In a function call, keyword arguments must follow positional arguments. All the keyword arguments passed must match one of the arguments accepted by the function (e.g. actor is not a valid argument for the parrot function), and their order is not important. This also includes non-optional arguments

*args and **kwargs contain arguments which do not match an existing formal parameter, so once you defined spam=None in the argument list, it won't be passed to **kwargs anymore:

When a final formal parameter of the form name is present, it receives a dictionary (see Mapping Types — dict) containing all keyword arguments **except for those corresponding to a formal parameter. This may be combined with a formal parameter of the form *name (described in the next subsection) which receives a tuple containing the positional arguments beyond the formal parameter list. (*name must occur before **name.)

The closest approximation to your (impossible) syntax would be something like

def foo(my_keyword0=None, my_keyword1='default', numbers=None, **kwargs)

which you would use as

foo(my_keyword0="bar", my_keyword1="baz", numbers=(1,2,3), spam='eggs')
foo("bar", "baz", numbers=(1,2,3))
foo("bar", "baz", (1,2,3))
foo("bar", "baz", (1,2,3), spam="eggs")
foo(numbers=(1,2,3))
foo(spam="eggs")
etc.

which, arguably, may be more readable and less surprising than your original idea.