Is NumPy any faster than default python when iterating over a list? [closed]

Question

I have a project where data is stored in a 2-dimensional array and each needs to be analyzed. I need to iterate through it. I just learned NumPy and was considering using it for this project, but after running some code I found that it ended up slower than just using a for loop.

import numpy
import time

list_ = [[num + 1 for num in range(5)] for lst in range(1000000)]
array = numpy.array([[num + 1 for num in range(5)] for lst in range(1000000)])

start = time.perf_counter()
count = 0
for lst in list_:
    for num in lst:
        count += 1
elapsed = time.perf_counter() - start
print(f"A default python list took {elapsed:0.5f} seconds to get through {count} elements.")

start = time.perf_counter()
count = 0
for num in numpy.nditer(array):
    count += 1
elapsed = time.perf_counter() - start
print(f"A numpy array took {elapsed:0.5f} seconds to get through {count} elements.")

The for loop takes on average ~0.5 seconds, while NumPy takes ~0.67 seconds. My run-through of NumPy syntax was somewhat surface level, so is there something I'm missing here that could run faster than the conventional for loop?

Answer 1

The previous answers pointed out in great detail how much faster NumPy is for particular tasks. I would like to add a holistic viewpoint on lists and NumPy arrays.

TL;DR: NumPy is fast whenever numerical computation of array-like objects is involved, sometimes by a factor of 10-100. However, looping through an array (instead of leveraging vectorization) as in your case can slow down the code so much that the computational benefit vanishes. Additionally, setting up a np.array or leveraging a NumPy function can introduce overhead when applied to a mere scalar that counteracts performance. Andrej Karpathy pointed this out in an infamous tweet.

Why and When NumPy is faster

On the one hand, NumPy arrays are densely packed, same-type data structures that leverage nearness in memory. On the other hand, NumPy operations are implemented in lightning-fast C. In addition, NumPy uses hardware-specific trickery (e.g. Intel's MKL) for accelerated vectorized operations on the CPU. This post goes into further detail on that.

I came up with four scenarios to test this. These are far from exhaustive but give a little bit of an idea. Note that the plot shows runtime on a logarithmic scale.

Initialize a list/an array

• (0) exclusively with 1 entries: NumPy is drastically faster in creating a static array of 1s
• (1) index-dependent entries: NumPy is slower as the list comprehension is used to initialize the np.array

(2) Inner product: This is where NumPy shines as it was specifically set up for such vector/matrix operations. Since the inner product (and, more generally, matrix multiplication) is at the heart of many scientific computations, NumPy is king.

(3) Reduction (example: maximum): Reducing/aggregating a list or an array to a single scalar usually works faster with NumPy even though min(), max(), sum() etc. are built-in functions in Python.

map: apply functions to the list/array

• (4) sqrt
• (5) defined function f: in this particular case, list comprehension turned out to be faster. However, the discussion is more complicated on how to apply functions efficiently.

Stemming from the code below that ran on a 2.8 GHz Quad-Core Intel i7.

import numpy as np
import timeit
import math

N = 1e6

def f(x):
    return 0.5*x**3 - 1.0/(x + 3.0)

# initialization
%timeit v1_l = [1 for i in range(N)]
%timeit v1_n = np.ones(N)

%timeit v2_l = [i**2 for i in range(N)]
%timeit v2_n = np.array([i**2 for i in range(N)])

# inner product
%timeit sum([v1_i * v1_j for (v1_i, v1_j) in zip(v1_l, v2_l)])
%timeit np.inner(v1_n, v2_n)

# reduction 
%timeit max(v2_l)
%timeit np.max(v2_n)

# map
%timeit [math.sqrt(x) for x in v2_l]
%timeit np.sqrt(v2_n)

%timeit [f(x) for x in v2_l]
%timeit np.fromiter((f(x) for x in v2_n), v2_n.dtype, count=len(v2_n))

What else to consider...

If you consistently deal with loops, consider Cython. In case you keep dealing with k-dimensional array-like structures, stay with NumPy. Chances are that numerical subsequent operations outweigh the initial performance disadvantage you might perceive.