Closures, Partials and Decorators

Question

I am confused between the three. I understand that closures are functions returned by another functions and have access to the local variables form an enclosing scope

example:

def add_nums(one):
    def adder(two):
        return one+two
    return adder

a_10 = add_nums(10)
print a_10(5)
15

Here, adder is a closure.

But, is this also not an example of a partial

from functools import partial
a_10 = partial(add_nums, 10)
print a_10()(5)
15

What is the difference between the two?

Also, decorators are used to add functionality to functions.

def add_nums(one):
    def adder(two):
        print "foo, bar"
        return one+two
    return adder

a_10 = add_nums(10)
print a_10(5)
foo, bar
15

What is the difference between all the three?

Answer 1

I think you're confusing purpose with implementation. Creating a closure is a technique for doing things. You can do a bunch of things with closures.

On the other hand, partial and decorator are particular purposes. Maybe they use closures. Maybe they don't. That's an implementation detail, and you don't need to worry about it. What's important is that they achieve the result you want.

Consider a partial. (Ignoring **kwargs.) One way to create it would be to use a closure:

def partial(f, *args):
    def pf(*rest):
        return f(*args, *rest)
    return pf

But it doesn't have to be that way. For example:

class Partial:
    def __init__(self, func, args):
        self.args = args
        self.func = func

    def __call__(self, *rest):
        f = self.func
        args = self.args
        return f(*args, *rest)

def partial(f, *args):
    return Partial(f, args)

There's no closure here, just a variable that holds a reference to the other variables. But I get the partial behavior, so who cares?

The same is true for decorators. A decorator might be a closure, or it might not. For example, one recent question involved running str.format on a function's __doc__ strings. That was just a case of accepting a function object, modifying the __doc__ attribute, and returning the same object. Clearly, there's no closure involved in that.

Answer 2

Short answer: closure is the mechanism, while functools.partial and decorators are typical uses of that mechanism.

The key difference between a closure and more typical namespaces is that the names and values in the "closed-over" namespace don't vanish when control flow leaves the top-level function. They're preserved in a mini-namespace associated with one instance of the inner function, and which survives as long as that instance does.

functools.partial uses that ability to "remember" some arguments to a pre-existing function. Decorators also typically use that ability for the same reasons.

Note that a decorator is more flexible than that. Any callable that takes one parameter and returns something can serve as a decorator. It doesn't have to make use of Python closures, or even return a function. Many decorators return a callable object instead. (As an aside, a closure and inner function can be simulated using an object for the namespace and a method for the inner function.)

Whatever the decorator returns is assigned to the name the decorated function would have had. (Decorators with parentheses after them, like @decorator('args') ... are slightly more complicated.) The typical decorator syntax:

@decorator
def function():
    pass

...is just a shorthand for "define, then decorate and reassign":

def function():
    pass
function = decorator(function)

For an extreme (and mostly useless) example:

def decorator5(__):
   return 5

@decorator5
def square(x):
    return x * x

print(square)  # Prints 5 --- the function is gone.
square(10)     # TypeError: 'int' object is not callable