Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does the inaccessible `.0` variable in `locals()` affect memory or performance?

Tags:

python

I maintain a project that has a function definition similar to this:

def f(a, (b1, b2), c):
    print locals()

While debugging the code I discovered that a .1 key appeared in locals(), with the value (b1, b2). A quick check revealed that a function definition like the following:

def f((a1, a2)):
    print locals()

will have a .0 key in locals() with the value (a1, a2). I was surprised by this behavior, but could find no information in the Python documentation.

My questions are: do these otherwise-inaccessible positional variables affect memory or performance? Are they documented anywhere? What purpose do they serve?

The project in question is feedparser, which is SAX-based and could potentially have dozens or hundreds of function calls that would be affected by this behavior.

like image 904
Kurt McKee Avatar asked Jan 08 '12 07:01

Kurt McKee


2 Answers

so pep 3113, as artur gaspar points out, contains a full answer. It also lists a whole bunch of reasons why this probably isn't a great pattern to follow. One of these you discovered in the annoying side effects of debugging. A bigger one is I think that your code will break transitioning to python3, but I'm not sure/still am on 2.7 personally.

I wanted to play with what happens. Looking as some disassembled bytecodes we can see what happens with these three functions (spoiler: foo and bar have identical bytecodes):

from dis import dis

def foo(a, (b, c) ,d):
    return a + b + c + d

def bar(a, b_c, d):
    b, c = b_c
    return a + b + c + d

def baz(a, b, c, d):
    return a + b + c + d

print '\nfoo:'
dis(foo)
print '\nbar:'
dis(bar)
print '\nbaz:'
dis(baz)

Yields:

foo:
  3           0 LOAD_FAST                1 (.1)
              3 UNPACK_SEQUENCE          2
              6 STORE_FAST               3 (b)
              9 STORE_FAST               4 (c)

  4          12 LOAD_FAST                0 (a)
             15 LOAD_FAST                3 (b)
             18 BINARY_ADD          
             19 LOAD_FAST                4 (c)
             22 BINARY_ADD          
             23 LOAD_FAST                2 (d)
             26 BINARY_ADD          
             27 RETURN_VALUE        


bar:
  7           0 LOAD_FAST                1 (b_c)
              3 UNPACK_SEQUENCE          2
              6 STORE_FAST               3 (b)
              9 STORE_FAST               4 (c)

  8          12 LOAD_FAST                0 (a)
             15 LOAD_FAST                3 (b)
             18 BINARY_ADD          
             19 LOAD_FAST                4 (c)
             22 BINARY_ADD          
             23 LOAD_FAST                2 (d)
             26 BINARY_ADD          
             27 RETURN_VALUE        


baz:
 11           0 LOAD_FAST                0 (a)
              3 LOAD_FAST                1 (b)
              6 BINARY_ADD          
              7 LOAD_FAST                2 (c)
             10 BINARY_ADD          
             11 LOAD_FAST                3 (d)
             14 BINARY_ADD          
             15 RETURN_VALUE        

As you can see. foo and bar are identical, while baz skips the unpacking. So yes, this will affect performance a bit, but only as long as tuple unpacking takes, which should be negligible in everything except very small functions and toy examples (like this one ;P)

like image 171
stein Avatar answered Nov 14 '22 21:11

stein


Yes, they do affect performance.

>>> import timeit
>>> 
>>> 
>>> def function_1(a, (b1, b2), c):
...     locals()
... 
>>> def function_2(a, b1, b2, c):
...     locals()
... 
>>> 
>>> object_1 = object()
>>> object_2 = object()
>>> object_3 = object()
>>> tuple_of_objects_2_and_3 = (object_2, object_3)
>>> object_4 = object()
>>> 
>>> n = 100000000
>>> 
>>> time_1 = timeit.timeit(lambda: function_1(object_1, tuple_of_objects_2_and_3, 
...                                           object_4),
...                        number=n)
>>> time_2 = timeit.timeit(lambda: function_2(object_1, object_2, object_3, 
...                                           object_4), 
...                        number=n)
>>> 
>>> print(time_1, time_2)
(170.2440218925476, 151.92010402679443)

About their documentation or purpose, I don't know.

like image 43
Artur Gaspar Avatar answered Nov 14 '22 21:11

Artur Gaspar