Does Python optimize function calls from loops?

Question

Say, I have a code which calls some function millions time from loop and I want the code to be fast:

def outer_function(file):
    for line in file:
        inner_function(line)

def inner_function(line):
    # do something
    pass

It's not necessarily a file processing, it could be for example a function drawing point called from function drawing line. The idea is that logically these two have to be separated, but from performance point of view they should act together as fast as possible.

Does Python detects and optimizes such things automatically? If not - is there a way to give it a clue to do so? Use some additional external optimizer maybe?...

Answer 1

Python does not inline function calls, because of its dynamic nature. Theoretically, inner_function can do something that re-binds the name inner_function to something else - Python has no way to know at compile time this might happen. For example:

def func1():
    global inner_func
    inner_func = func2
    print 1

def func2():
    print 2

inner_func = func1

for i in range(5):
    inner_func()

Prints:

1
2
2
2
2

You may think this is horrible. Then, think again - Python's functional and dynamic nature is one of its most appealing features. A lot of what Python allows comes at the cost of performance, and in most cases this is acceptable.

That said, you can probably hack something together using a tool like byteplay or similar - disassemble the inner function into bytecode and insert it into the outer function, then reassemble. On second thought, if your code is performance-critical enough to warrant such hacks, just rewrite it in C. Python has great options for FFI.

---

This is all relevant to the official CPython implementation. A runtime-JITting interpreter (like PyPy or the sadly defunct Unladen Swallow) can in theory detect the normal case and perform inlining. Alas, I'm not familiar enough with PyPy to know whether it does this, but it definitely can.