How can I determine the function in which a closure was created?

Question

This code in Python 2.7 creates a closure around func, enclosing the par variable:

def creator(par):
    def func(arg):
        return par + arg
    return func

It can by used like so:

f = creator(7)
f(3)               # returns 10

At runtime, is there a way to get name of the function in which the closure was defined? That is: having only access to the f variable, can I get information that f closure was defined inside creator function?

I am using Python 2.7.

Answer 1

You can use __qualname__ for this, which represents the qualified function name

def creator(par):
    def func(arg):
        return par + arg
    return func

>>> f = creator(7)
>>> f.__qualname__
'creator.<locals>.func'

Answer 2

I suspect it can't be done in Python 2.7, at least not directly. Here's the pretty-printed contents of your function f, minus some obviously-unrelated entries:

>>> pprint({key: getattr(f, key) for key in dir(f)})
{'__call__': <method-wrapper '__call__' of function object at 0x7f9d22aefb18>,
 '__class__': <type 'function'>,
 '__closure__': (<cell at 0x7f9d22af1fd8: int object at 0x1311128>,),
 '__code__': <code object func at 0x7f9d22d3b230, file "<stdin>", line 2>,
 '__defaults__': None,
 '__dict__': {},
 '__doc__': None,
 '__module__': '__main__',
 '__name__': 'func',
 'func_closure': (<cell at 0x7f9d22af1fd8: int object at 0x1311128>,),
 'func_code': <code object func at 0x7f9d22d3b230, file "<stdin>", line 2>,
 'func_defaults': None,
 'func_dict': {},
 'func_doc': None,
 'func_globals': {'__builtins__': <module '__builtin__' (built-in)>,
                  '__doc__': None,
                  '__name__': '__main__',
                  '__package__': None,
                  'creator': <function creator at 0x7f9d22aefaa0>,
                  'f': <function func at 0x7f9d22aefb18>,
                  'pprint': <function pprint at 0x7f9d22aefc08>},
 'func_name': 'func'}

The only interesting keys are func_closure (which is __closure__) and func_code (which is __code__), but neither help.

The closure is a tuple of cell objects, each of which contains the value of a key-value pair in the closed-over environment. There is only one value in f.func_closure, that of the par variable:

>>> repr(f.func_closure[0])
<cell at 0x7f9d22af1fd8: int object at 0x1311128>
>>> f.func_closure[0].cell_contents
7

A cell does not contain a reference to the creator of the closure, the function using the closure, or even the enclosed environment itself. (Elements of the enclosed environment appear to be retrieved based on their position in the tuple of cell objects.)

The function code object comes closer, but also doesn't name its creator. Minus the obviously irrelevant entries, it contains:

>>> pprint({k: getattr(f.func_code, k) for k in dir(f.func_code)})
{'__class__': <type 'code'>,
 '__doc__': 'code(argcount, nlocals, stacksize, flags, codestring, constants, names,\n      varnames, filename, name, firstlineno, lnotab[, freevars[, cellvars]])\n\nCreate a code object.  Not for the faint of heart.',
 'co_argcount': 1,
 'co_cellvars': (),
 'co_code': '\x88\x00\x00|\x00\x00\x17S',
 'co_consts': (None,),
 'co_filename': '<stdin>',
 'co_firstlineno': 2,
 'co_flags': 19,
 'co_freevars': ('par',),
 'co_lnotab': '\x00\x01',
 'co_name': 'func',
 'co_names': (),
 'co_nlocals': 1,
 'co_stacksize': 2,
 'co_varnames': ('arg',)}

This contains the name of the closed-over variable ('co_freevars': ('par',),), and the name of the function inside creator ('co_name': 'func',) but not the name or any reference to the outer function.

Partial Solution

There is a way to identify the outer function of an enclosed function, if you have a reference to both. The creating function's function code object will contain a reference to the closed-over function's code object:

>>> pprint({k: getattr(creator.func_code, k) for k in dir(creator.func_code)})
{'__class__': <type 'code'>,
 '__doc__': 'code(argcount, nlocals, stacksize, flags, codestring, constants, names,\n      varnames, filename, name, firstlineno, lnotab[, freevars[, cellvars]])\n\nCreate a code object.  Not for the faint of heart.',
 'co_argcount': 1,
 'co_cellvars': ('par',),
 'co_code': '\x87\x00\x00f\x01\x00d\x01\x00\x86\x00\x00}\x01\x00|\x01\x00S',
 'co_consts': (None,
               <code object func at 0x7f9d22d3b230, file "<stdin>", line 2>),
 'co_filename': '<stdin>',
 'co_firstlineno': 1,
 'co_flags': 3,
 'co_freevars': (),
 'co_lnotab': '\x00\x01\x0f\x02',
 'co_name': 'creator',
 'co_names': (),
 'co_nlocals': 2,
 'co_stacksize': 2,
 'co_varnames': ('par', 'func')}

You can determine that creator is the source of f, because the tuple creator.func_code.co_consts contains a reference to f.func_code:

>>> f.func_code in creator.func_code.co_consts
True
>>> f.func_code is creator.func_code.co_consts[1]
True

The same code object is used by every function returned by creator (their differences stored in cell objects, not code objects):

>>> g = creator(10)
>>> g.func_code is f.func_code is creator.func_code.co_consts[1]
True

So, if you can narrow the potential sources down to say, values in globals() or in dir(some_class), you can test each of them to see if it's the "parent" of f:

def is_creator(f, contender):
    target = f.func_code
    try:
        constants = contender.func_code.co_consts
    except AttributeError:
        return False
    for constant in constants:
        if constant is target:
            return True
    return False

def find_creators(f, contenders):
    for contender in contenders:
        if is_creator(f, contender):
            yield contender
    return

>>> is_creator(f, creator)
True
>>> is_creator(g, creator)
True

>>> is_creator(f, max)
False
>>> is_creator(f, g)
False
>>> is_creator(f, 'Seriously?')
False
>>> is_creator(f, None)
False

>>> list(find_creators(f, globals().values()))
[<function creator at 0x7f9d22aefaa0>]

>>> builtins = [getattr(__builtins__, s) for s in dir(__builtins__)]
>>> list(find_creators(f, builtins))
[]

This kind of sucks, really, because it doesn't point you to the creator, it just identifies the creator if you've already found it. Also, it can be fooled if someone uses creator.__code__ to build an imposter:

def impostor(bogus):
    def need_a_free_variable_in_impostors_func_code(unused):
        return bogus - unused
    return need_a_free_variable_in_impostors_func_code

>>> creator(3)(7)
10
>>> impostor(3)(7)
-4
>>> is_creator(f, impostor)
False

>>> impostor.__code__ = creator.__code__
>>> impostor(3)(7)
10
>>> is_creator(f, impostor)
True
>>> list(find_creators(f, globals().values()))
[<function creator at 0x7f9d22aefaa0>, <function impostor at 0x7f9d1bf7f8c0>]

Questionable Improvements

There are other clues, once a potential creator is found, but they don't really constitute proof. Examples include:

The name of the closed-over variable 'par' appears in both f as a free variable, and in creator as a "cell" variable:

>>> f.func_code.co_freevars[0] in creator.func_code.co_cellvars
True

The name of f (which is 'func', not 'f') appears in the creator's function code object. (Function code objects are immutable, so f.func_code.co_name must be the original name assigned to f when it was created. f.__name__ could have been reassigned since then. So could f.func_code --- the whole code object --- but that's nowhere near as common.)

>>> f.func_code.co_name in creator.func_code.co_varnames
True

Because function definitions can be deeply nested --- meaning different free variables in the innermost function could be defined (recorded in co_cellvars) in different outer functions --- I don't think adding checks for these would make is_creator any "smarter".