Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is a constant list used in a loop constructed/deleted with each pass?

Tags:

python

Will the following snippet create and destroy the list of constants on each loop, incurring whatever (albeit small) overhead this implies, or is the list created once?

for i in <some-type-of-iterable>:
    if i in [1,3,5,18,3457,40567]:
        print(i)

I am asking about both the Python "standard", such one as exists, and about the common CPython implementation.
I am aware that this example is contrived, as well as that trying to worry about performance using CPython is silly, but I am just curious.

like image 208
Baruch Avatar asked May 24 '16 18:05

Baruch


1 Answers

This depends on the python implementation and version and how the "constant lists" are used. On Cpython2.7.10 with your example, it looks like the answer is that the list in the condition of the if statement is only created once...

>>> def foo():
...   for i in iterable:
...     if i in [1, 3, 5]:
...       print(i)
... 
>>> import dis
>>> dis.dis(foo)
  2           0 SETUP_LOOP              34 (to 37)
              3 LOAD_GLOBAL              0 (iterable)
              6 GET_ITER            
        >>    7 FOR_ITER                26 (to 36)
             10 STORE_FAST               0 (i)

  3          13 LOAD_FAST                0 (i)
             16 LOAD_CONST               4 ((1, 3, 5))
             19 COMPARE_OP               6 (in)
             22 POP_JUMP_IF_FALSE        7

  4          25 LOAD_FAST                0 (i)
             28 PRINT_ITEM          
             29 PRINT_NEWLINE       
             30 JUMP_ABSOLUTE            7
             33 JUMP_ABSOLUTE            7
        >>   36 POP_BLOCK           
        >>   37 LOAD_CONST               0 (None)
             40 RETURN_VALUE        

Notice: 16 LOAD_CONST 4 ((1, 3, 5))

Python's peephole optimizer has turned our list into a tuple (thanks python!) and stored it as a constant. Note that the peephole optimizer can only do these transforms on objects if it knows that you as the programmer have absolutely no way of getting a reference to the list (otherwise, you could mutate the list and change the meaning of the code). As far as I'm aware, they only do this optimization for list, set literals that are composed of entirely constants and are the RHS of an in operator. There might be other cases that I'm not aware of (dis.dis is your friend for finding these optimizations).

I hinted at it above, but you can do the same thing with set-literals in more recent versions of python (in python3.2+, the set is converted to a constant frozenset). The benefit there is that set/frozenset have faster membership testing on average than list/tuple.

like image 121
mgilson Avatar answered Sep 20 '22 18:09

mgilson