I've discovered surprising python behaviour while had investigating thread Why is reading lines from stdin much slower in C++ than Python?.
If I run simple python code from that thread
#!/usr/bin/env python
from __future__ import print_function
import time
import sys
count = 0
start_time = time.time()
for line in sys.stdin:
count += 1
delta_sec = time.time() - start_time
if delta_sec >= 0:
lines_per_sec = int(round(count/delta_sec))
print("Read {0:n} lines in {1:.2f} seconds. LPS: {2:n}".format(count, delta_sec, lines_per_sec))
it works with speed 11.5M LPS, and when I decompose the whole script into single function
#!/usr/bin/env python
from __future__ import print_function
import time
import sys
def test(input):
count = 0
start_time = time.time()
for line in input:
count += 1
delta_sec = time.time() - start_time
if delta_sec >= 0:
lines_per_sec = int(round(count/delta_sec))
print("Read {0:n} lines in {1:.2f} seconds. LPS: {2:n}".format(count, delta_sec, lines_per_sec))
if __name__ == "__main__":
test(sys.stdin)
code speeds up to 23M LPS.
Why this simple refactoring makes my code 2 times faster?
I've run my tests with python2.7 on Ubuntu 13.10.
Watching into bytecode helped me to answer this question. Byte code for working part of the first script is:
10 58 SETUP_LOOP 27 (to 88)
61 LOAD_NAME 3 (sys)
64 LOAD_ATTR 6 (stdin)
67 GET_ITER
>> 68 FOR_ITER 16 (to 87)
71 STORE_NAME 7 (line)
11 74 LOAD_NAME 4 (count)
77 LOAD_CONST 4 (1)
80 INPLACE_ADD
81 STORE_NAME 4 (count)
84 JUMP_ABSOLUTE 68
>> 87 POP_BLOCK
And byte code for corresponding part of second script is:
12 18 SETUP_LOOP 24 (to 45)
21 LOAD_FAST 0 (input)
24 GET_ITER
>> 25 FOR_ITER 16 (to 44)
28 STORE_FAST 3 (line)
13 31 LOAD_FAST 1 (count)
34 LOAD_CONST 2 (1)
37 INPLACE_ADD
38 STORE_FAST 1 (count)
41 JUMP_ABSOLUTE 25
>> 44 POP_BLOCK
I see that actual difference between this codes is LOAD_NAME vs LOAD_FAST and STORE_NAME vs STORE_FAST opcodes using. Documentation http://docs.python.org/2.7/library/dis.html#opcode-LOAD_FAST says that LOAD_FAST makes lookup using only indexes, while LOAD_NAME lookups variable by string name. And the first approach is two times faster.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With