Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Most efficient way to map function over numpy array

What is the most efficient way to map a function over a numpy array? The way I've been doing it in my current project is as follows:

import numpy as np   x = np.array([1, 2, 3, 4, 5])  # Obtain array of square of each element in x squarer = lambda t: t ** 2 squares = np.array([squarer(xi) for xi in x]) 

However, this seems like it is probably very inefficient, since I am using a list comprehension to construct the new array as a Python list before converting it back to a numpy array.

Can we do better?

like image 249
Ryan Avatar asked Feb 05 '16 02:02

Ryan


People also ask

Does NumPy have a map function?

Method 1: numpy.The numpy. vectorize() function maps functions on data structures that contain a sequence of objects like NumPy arrays.

How can I speed up my NumPy operation?

By explicitly declaring the "ndarray" data type, your array processing can be 1250x faster. This tutorial will show you how to speed up the processing of NumPy arrays using Cython. By explicitly specifying the data types of variables in Python, Cython can give drastic speed increases at runtime.

Is NumPy array faster than dictionary?

Also as expected, the Numpy array performed faster than the dictionary.


1 Answers

I've tested all suggested methods plus np.array(map(f, x)) with perfplot (a small project of mine).

Message #1: If you can use numpy's native functions, do that.

If the function you're trying to vectorize already is vectorized (like the x**2 example in the original post), using that is much faster than anything else (note the log scale):

enter image description here

If you actually need vectorization, it doesn't really matter much which variant you use.

enter image description here


Code to reproduce the plots:

import numpy as np import perfplot import math   def f(x):     # return math.sqrt(x)     return np.sqrt(x)   vf = np.vectorize(f)   def array_for(x):     return np.array([f(xi) for xi in x])   def array_map(x):     return np.array(list(map(f, x)))   def fromiter(x):     return np.fromiter((f(xi) for xi in x), x.dtype)   def vectorize(x):     return np.vectorize(f)(x)   def vectorize_without_init(x):     return vf(x)   b = perfplot.bench(     setup=np.random.rand,     n_range=[2 ** k for k in range(20)],     kernels=[         f,         array_for,         array_map,         fromiter,         vectorize,         vectorize_without_init,     ],     xlabel="len(x)", ) b.save("out1.svg") b.show() 
like image 117
Nico Schlömer Avatar answered Sep 22 '22 10:09

Nico Schlömer