Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Performance difference between filling existing numpy array and creating a new one

Tags:

python

numpy

In iterative algorithms, it is common to use large numpy arrays many times. Frequently the arrays need to be manually "reset" on each iteration. Is there a performance difference between filling an existing array (with nans or 0s) and creating a new array? If so, why?

like image 383
Sean Mackesey Avatar asked Jul 19 '15 07:07

Sean Mackesey


People also ask

Is appending to NumPy array faster than list?

array(a) . List append is faster than array append .

How can I make NumPy array faster?

By explicitly declaring the "ndarray" data type, your array processing can be 1250x faster. This tutorial will show you how to speed up the processing of NumPy arrays using Cython. By explicitly specifying the data types of variables in Python, Cython can give drastic speed increases at runtime.

Why NumPy array operations are faster than looping?

NumPy Arrays are faster than Python Lists because of the following reasons: An array is a collection of homogeneous data-types that are stored in contiguous memory locations. On the other hand, a list in Python is a collection of heterogeneous data types stored in non-contiguous memory locations.

Is NumPy array memory efficient?

NumPy arrays are faster and more compact than Python lists. An array consumes less memory and is convenient to use. NumPy uses much less memory to store data and it provides a mechanism of specifying the data types. This allows the code to be optimized even further.


1 Answers

The answer depends on the size of your arrays. While allocating a new memory region takes nearly a fixed amount of time, the time to fill this memory region grows linear with size. But, filling a new allocated memory with numpy.zeros is nearly twice as fast, as filling an existing array with numpy.fill, and three times faster than item setting x[:] = 0.

So on my machine, filling vectors with less than 800 elements is faster than creating new vectors, with more than 800 elements creating new vectors gets faster.

like image 194
Daniel Avatar answered Oct 12 '22 06:10

Daniel