What's the pythonic way to pass arguments between functions?

Question

I have some arguments taken from the user and passed along function to function (each function in a different class), until it eventually gets to a function that does some processing and then the solution is returned up the chain. Up the chain, the functions become more and more abstract merging results from multiple runs of the lower functions.

Where should I use *args and **kwargs?

I think *args and *kwargs can be used for every function where the function doesn't use the arguments explicitly. But, the actual arguments need to be defined at the top_level so that the user knows what the function expects.

Where should I define what the inputs mean?

I think they should be defined at the top_level because that's the one the end-user might want to see the documentation for.

Where should I define what default values?

Again, I think they should be defined at the top_level because that's the one the end-user interacts with.

This is a simple example to demonstrate the passing of the arguments, where I haven't shown how the functions become more and more abstract or how they interact with different classes, as I felt it was unnecessary detail.

def top_level(a=1, b=1, c=1, d=1, e=1):
    """ Compute sum of five numbers.
    :param a: int, a
    :param b: int, b
    :param c: int, c
    :param d: int, d
    :param e: int, e
    :return: int, sum
    """
    return mid_level(a, b, c, d, e)


def mid_level(*args, **kwargs):
    return bottom_level(*args, **kwargs)


def bottom_level(a, b, c, d, e):
    return a + b + c + d + e

print top_level(1, 2, 3)
8

Is there a Python convention for passing arguments like this?

Answer 1

I'm not going to answer your question because it would be like answering the question "what's the best way to use a screwdriver to tighten a nut?". I.e. I do not believe that the tools you are asking for guidance with (*args and **kwargs) are designed to solve the problem you want to solve.

Instead I'll answer this question: "how do I associate some data with a set of functions?", and the answer to that is clearly Use Classes.

Welcome to object-oriented programming. I think you're going to enjoy it!

This is a very basic example of what I mean, but it was hard to know exactly what you wanted from your example since it was simple, but the basic principle is encapsulate your data in a class, and then operate on it using the class's methods.

• You can then call between methods in the class without needing to pass loads of arguments around all the time (such as the .calculate() method below), which you don't know whether the top layer will need or a bottom layer.
• You can just document the parameters in one place, the __init__ method.
• You can customize through subclassing transparently to the code (because if you override a method in a subclass, it can still be used by the more generic superclass), as I've done for the .reduce(x, y) method below.

Example:

class ReductionCalculator:

    def __init__(self, *args):
        self.args = args

    def calculate(self):

        start = self.args[0]
        for arg in self.args[1:]:
            start = self.reduce(start, arg)

        return start


class Summer(ReductionCalculator):

    def reduce(self, x, y):
        return x + y


class Multiplier(ReductionCalculator):

    def reduce(self, x, y):
        return x * y

summer = Summer(1, 2, 4)
print('sum: %d' % (summer.calculate(),))
multiplier = Multiplier(1, 2, 4)
print('sum: %d' % (multiplier.calculate(),))

Answer 2

How about this approach: create a class, call it AllInputs, that represents the collection of all the "arguments taken from the user." The only purpose of this class is to serve as a container for a set of values. One instance of this class gets initialized, of course, at the top level of the program.

class AllInputs:
   def __init__(self,a=1, b=1, c=1, d=1, e=1):
    """ Compute sum of five numbers.
    :param a: int, a
    :param b: int, b
    :param c: int, c
    :param d: int, d
    :param e: int, e
    """
    self.a = a
    self.b = b
    self.c = c
    self.d = d
    self.e = e

This object, call it all_inputs, is now passed as the single argument to all of the functions in your example. If a function doesn't use any of the fields in the object, that's fine; it just passes it along to the lower-level function where the real work gets done. To refactor your example, you would now have:

def top_level(all_inputs):
    """ Compute sum of all inputs
    :return: int, sum
    """
    return mid_level(all_inputs)


def mid_level(all_inputs):
    return bottom_level(all_inputs)


def bottom_level(all_inputs):
    return (all_inputs.a + all_inputs.b + all_inputs.c + 
      all_inputs.d + all_inputs.e)

all_inputs = AllInputs(1, 2, 3)
print top_level(all_inputs)
8

I don't know if this is "Pythonic" or "non-Pythonic" and I don't care. I think it's a good programming idea to group together the data that the program will use. The initialization process, which combines default values with others taken from the user, is centralized in one place where it's easy to understand. It's reasonably self-documenting. You say the function calls are distributed across several classes, and that's no problem. The function calls are clean and the program flow is easy to follow. There is potential for optimization by placing some of the calculation inside AllInputs so you can avoid duplicating code.

What I don't like in your example (and I think you don't like it either, or you probably wouldn't have asked the question in the first place) is how it uses the *args syntax. When I see that syntax, I take it as a hint that all the arguments have the same semantic meaning, like in the standard library function os.path.join. In your application, if I understand the question, the low-level functions require the argument list to be in a specific order and have specific meanings (your example doesn't reflect that but the text suggests it). It's confusing to see arguments that get passed into a function as *args and then, at a lower level, their specific names and meanings appear once again. Grouping them into a single object makes it clear what's going on.

Answer 3

This isn't the most common pattern, but I've seen it for command line programs that have levels of nested commands: sub-commands, sub-sub-commands and so on. That's a model where "upper" level functions may be more or less dispatchers and not have information about what parameters are needed by the sub-functions within a given route. The purest scenario for this model is when the sub-commands are plugins and the "upper" layers have literally no information about the sub-functions, other than a calling convention the plug-ins are expected to adhere to.

In these cases, I'd argue the pythonic way is to pass parameters from higher-level to lower-level functions, and let the worker level decide which are useful. The range of possible parameters would be defined in the calling convention. This is pythonic on the basis of DRY -- don't repeat yourself. If the low-level / worker function defines what inputs are required or optional, it would often make sense to not repeat this information at the higher levels.

The same could be said for any inversion-of-control flow design, not just CLI applications w/ plug-ins. There are many application designs where I wouldn't use this approach, but it works here.

An input's meaning must be set at the topmost level it can arise in -- as an interface spec to lower levels (a convention, not programmatic). Otherwise the inputs would have no semantic meaning.

If an input can be used by multiple sub-functions, i.e. there's a chaining or pipeline concept in the control flow, then an input's default will also need to be defined at the topmost level for the input.