Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is Python's built-in sum much slower than manual summation?

Here is my simple code example:

import time

t0 = time.time()
s = 0
for i in range(1000000):
    s += i
t1 = time.time()

print(s, t1 - t0)

t0 = time.time()
s = sum(i for i in range(1000000))
t1 = time.time()

print(s, t1 - t0)

On my computer (with Python 3.8) it prints:

499999500000 0.22901296615600586
499999500000 1.6930372714996338

So, doing += a million times is 7 times faster than calling sum? That is really unexpected. What is it doing?


Edit: I foolishly allowed a debugger to attach to the process and interfere with my measurements, which was in the end the cause of the slowness. With the debugger out, the measurements are no longer so unpredictable. As some of the answers are clearly showing, what I observed shuold not happen.

like image 237
zvone Avatar asked Jun 22 '20 12:06

zvone


People also ask

Is sum faster than for loop?

A key difference between R and many other languages is a topic known as vectorization. When you wrote the total function, we mentioned that R already has sum to do this; sum is much faster than the interpreted for loop because sum is coded in C to work with a vector of numbers.

Are python Builtins faster?

Use the Built-In FunctionsMany of Python's built-in functions are written in C, which makes them much faster than a pure python solution. Take a very simple task of summing a lot of numbers. We could loop through each number, summing as we go.

How is sum () implemented in python?

Python provides an inbuilt function sum() which sums up the numbers in the list. Syntax: sum(iterable, start) iterable : iterable can be anything list , tuples or dictionaries , but most importantly it should be numbers. start : this start is added to the sum of numbers in the iterable.


1 Answers

Let's use timeit for proper benchmarking and to make it easy to also compare different Python versions, let's run this in Docker containers:

so62514160.py

N = 1000000

def m1():
    s = 0
    for i in range(N):
        s += i

def m2():
    s = sum(i for i in range(N))

def m3():
    s = sum(range(N))

so62514160bench.sh

for image in python:2.7 python:3.6 python:3.7 python:3.8; do
    for fun in m1 m2 m3; do
        echo -n "$image" "$fun "
        docker run --rm -it -v $(pwd):/app -w /app -e PYTHONDONTWRITEBYTECODE=1 "$image" python -m timeit -s 'import so62514160 as s' "s.$fun()"
    done
done

results on my machine:

python:2.7 m1 10 loops, best of 3:  43.5 msec per loop
python:2.7 m2 10 loops, best of 3:  39.6 msec per loop
python:2.7 m3 100 loops, best of 3: 17.1 msec per loop
python:3.6 m1 10 loops, best of 3:  41.9 msec per loop
python:3.6 m2 10 loops, best of 3:  46 msec per loop
python:3.6 m3 100 loops, best of 3: 17.7 msec per loop
python:3.7 m1 5 loops, best of 5:   45 msec per loop
python:3.7 m2 5 loops, best of 5:   40.7 msec per loop
python:3.7 m3 20 loops, best of 5:  17.3 msec per loop
python:3.8 m1 5 loops, best of 5:   48.2 msec per loop
python:3.8 m2 5 loops, best of 5:   44.6 msec per loop
python:3.8 m3 10 loops, best of 5:  19.2 msec per loop

plot

enter image description here

like image 119
AKX Avatar answered Oct 14 '22 15:10

AKX