Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: Why does copying the function name to a local namespace result in a faster access

Tags:

From the code here: https://www.learnsteps.com/increasing-performance-python-code/

import datetime 
alist = [str(x) for x in range(100000000)]

print("\nStandard loop.") 
a = datetime.datetime.now() 
result = [] 
for item in alist: 
    result.append(len(item)) 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 

print("\nStandard loop with function name in local namespace.") 
a = datetime.datetime.now() 
result = [] 
fn = len 
for item in alist:
    result.append(fn(item))
b = datetime.datetime.now()
print((b-a).total_seconds())

print("\nUsing map.")
a = datetime.datetime.now()
result = list(map(len, alist))
b = datetime.datetime.now()
print((b-a).total_seconds())

print("\nUsing map with function name in local namespace.")
a = datetime.datetime.now() 
fn = len 
result = list(map(fn, alist)) 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 

print("\nList comprehension.") 
a = datetime.datetime.now() 
result = [len(i) for i in alist] 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 
print("\nList comprehension with name in local namespace.") 

a = datetime.datetime.now() 
fn = len 
result = [fn(i) for i in alist] 
b = datetime.datetime.now() 
print((b-a).total_seconds())

which produces this output:

Standard loop.
20.862797

Standard loop with function name in local namespace.
16.34087

Using map.
6.893764

Using map with function name in local namespace.
6.774654

List comprehension.
9.362831

List comprehension with name in local namespace.
10.007393

Can someone provide a better explanation than 'function lookups are costly' as to why creating a function prototype close to the use of the function is somehow faster? (This doesn't work for most functions, and usually only in tight loops, but why does this happen at all?)

like image 471
azazelspeaks Avatar asked Nov 02 '18 23:11

azazelspeaks


People also ask

What's the main difference between local and global namespaces?

Local Namespace: This namespace includes local names inside a function. This namespace is created when a function is called, and it only lasts until the function returns. Global Namespace: This namespace includes names from various imported modules that you are using in a project.

How does namespace work in Python?

Namespaces in Python. A namespace is a collection of currently defined symbolic names along with information about the object that each name references. You can think of a namespace as a dictionary in which the keys are the object names and the values are the objects themselves.

What is the difference between namespace and scope in Python?

A namespace is just a mapping from names to objects. A scope is a textual region of a Python program where a namespace is directly accessible.

Which feature in Python helps us to avoid namespace collision?

Scoping: Modules typically define a separate namespace, which helps avoid collisions between identifiers in different areas of a program. (One of the tenets in the Zen of Python is Namespaces are one honking great idea—let's do more of those!)


1 Answers

This is because name resolution starts from the local namespace first, and if it is not found locally, it would then be looked up in the next nearest enclosing code block, and then its next nearest enclosing code block, until the module code block, which is the global namespace, and if the name isn't found in the global namespace, then and only then will the interpreter look up the built-in names. This is why assigning the reference to the built-in name len to a global name fn would speed up the name resolution in your code example.

like image 83
blhsing Avatar answered Oct 13 '22 00:10

blhsing