Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Are Python inner functions compiled?

Tags:

python

So AFAIK in CPython, function definitions are compiled into function objects when executed at parse time. But what about inner functions? Do they get compiled into function objects at parse time or do they get compiled (or interpreted) every single time the function is called? Do inner functions incur any performance penalty at all?

like image 954
Y.H Wong Avatar asked May 16 '11 16:05

Y.H Wong


People also ask

What are Python inner functions?

Inner functions, also known as nested functions, are functions that you define inside other functions. In Python, this kind of function has direct access to variables and names defined in the enclosing function. Inner functions have many uses, most notably as closure factories and decorator functions.

Should you use inner functions Python?

Use an inner function if it is complex and you don't want to make it "public". Use a "private" method if you want to mark it hidden and uses members of the instance. Use a method if you want to make it "public" and uses members of the instance. Use a class method if it uses class members.

How nested function works in Python?

A function defined inside another function is called a nested function. Nested functions can access variables of the enclosing scope. In Python, these non-local variables are read-only by default and we must declare them explicitly as non-local (using nonlocal keyword) in order to modify them.

How do nested functions behave?

A nested function can access other local functions, variables, constants, types, classes, etc. that are in the same scope, or in any enclosing scope, without explicit parameter passing, which greatly simplifies passing data into and out of the nested function. This is typically allowed for both reading and writing.


2 Answers

To give a general explaination - assuming you have the following code in a module:

def outer(x=1):
    def inner(y=2):
        return x+y

When the file is parsed by python via compile(), the above text is turned into bytecode for how to execute the module. In the module bytecode, there are two "code objects", one for the bytecode of outer() and one for the bytecode inner(). Note that I said code objects, not functions - the code objects contain little more than the bytecode used by the function, and any information that could be known at compile time - such as the bytecode for outer() containing a ref to the bytecode for inner().

When the module actually loads, by evaluating the code object associated with the module, one thing which happens is an actual "function object" is created for outer(), and stored in the module's outer attribute. The function object acts as a collection of the bytecode and all context-relavant things that are needed to call the function (eg which globals dict it should pull from, etc) that can't be known at compile time. In a way, a code object is a template for a function, which is a template for execution of the actual bytecode with all variables filled in.

None of this involved inner()-as-a-function yet - Each time you actually get around to calling outer(), that's when a new inner() function object is created for that invocation of outer, which binds the already-created inner bytecode object to a list of globals, including the value of x as passed into that call to outer. As you can imagine, this is pretty fast, since no parsing is needed, just filling in a quick struct with some pointers to other already-existing objects.

like image 108
Eli Collins Avatar answered Oct 23 '22 11:10

Eli Collins


Easy test: the default arguments to a function are called once, at define time.

>>> def foo():
...     def bar(arg=count()):
...             pass
...     pass
...
>>> def count():
...     print "defined"
...
>>> foo()
defined
>>> foo()
defined

So yes: this is a minor (very very! minor) performance hit.

like image 9
Katriel Avatar answered Oct 23 '22 10:10

Katriel