Menu
I was wondering why list comprehension is so much faster than appending to a list. I thought the difference is just expressive, but it's not.
>>> import timeit >>> timeit.timeit(stmt='''\ t = [] for i in range(10000): t.append(i)''', number=10000) 9.467898777974142 >>> timeit.timeit(stmt='t= [i for i in range(10000)]', number=10000) 4.1138417314859 The list comprehension is 50% faster. Why?
610
List comprehensions are faster than for loops to create lists. But, this is because we are creating a list by appending new elements to it at each iteration.
One main benefit of using a list comprehension in Python is that it's a single tool that you can use in many different situations. In addition to standard list creation, list comprehensions can also be used for mapping and filtering. You don't have to use a different approach for each scenario.
It does slow down like you claimed. (0.03 seconds for the first iteration, and 0.84 seconds for the last... quite a difference.) Obviously, if you instantiate a list but don't append it to x , it runs way faster and doesn't scale up over time.
Map function is faster than list comprehension when the formula is already defined as a function earlier. So, that map function is used without lambda expression.
List comprehension is basically just a "syntactic sugar" for the regular for loop. In this case the reason that it performs better is because it doesn't need to load the append attribute of the list and call it as a function at each iteration. In other words and in general, list comprehensions perform faster because suspending and resuming a function's frame, or multiple functions in other cases, is slower than creating a list on demand.
Consider the following examples :
In [1]: def f1(): ...: l = [] ...: for i in range(5): ...: l.append(i) ...: ...: ...: def f2(): ...: [i for i in range(5)] ...: In [3]: import dis In [4]: dis.dis(f1) 2 0 BUILD_LIST 0 2 STORE_FAST 0 (l) 3 4 LOAD_GLOBAL 0 (range) 6 LOAD_CONST 1 (5) 8 CALL_FUNCTION 1 10 GET_ITER >> 12 FOR_ITER 14 (to 28) 14 STORE_FAST 1 (i) 4 16 LOAD_FAST 0 (l) 18 LOAD_METHOD 1 (append) 20 LOAD_FAST 1 (i) 22 CALL_METHOD 1 24 POP_TOP 26 JUMP_ABSOLUTE 12 >> 28 LOAD_CONST 0 (None) 30 RETURN_VALUE In [5]: In [5]: dis.dis(f2) 8 0 LOAD_CONST 1 (<code object <listcomp> at 0x7f397abc0d40, file "<ipython-input-1-45c11e415ee9>", line 8>) 2 LOAD_CONST 2 ('f2.<locals>.<listcomp>') 4 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (range) 8 LOAD_CONST 3 (5) 10 CALL_FUNCTION 1 12 GET_ITER 14 CALL_FUNCTION 1 16 POP_TOP 18 LOAD_CONST 0 (None) 20 RETURN_VALUE Disassembly of <code object <listcomp> at 0x7f397abc0d40, file "<ipython-input-1-45c11e415ee9>", line 8>: 8 0 BUILD_LIST 0 2 LOAD_FAST 0 (.0) >> 4 FOR_ITER 8 (to 14) 6 STORE_FAST 1 (i) 8 LOAD_FAST 1 (i) 10 LIST_APPEND 2 12 JUMP_ABSOLUTE 4 >> 14 RETURN_VALUE In [6]: You can see that on offset 18 in the first function we have an append attribute while there's no such thing in second function using list comprehension. All those extra bytecodes will make the appending approach slower and since in this case you'll have loading of the append attribute in each iteration, in the end it will make the code to take approximately twice as slower as the second function using only list comprehension.
73
Even factoring out the time it takes to lookup and load the append function, the list comprehension is still faster because the list is created in C, rather than built up one item at a time in Python.
# Slow timeit.timeit(stmt=''' for i in range(10000): t.append(i)''', setup='t=[]', number=10000) # Faster timeit.timeit(stmt=''' for i in range(10000): l(i)''', setup='t=[]; l=t.append', number=10000) # Faster still timeit.timeit(stmt='t = [i for i in range(10000)]', number=10000) If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
