Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Avoid copying when indexing a numpy arrays using lists

Tags:

python

numpy

Is there a simple way to index arrays using lists or any other collection so that no copy is made (just a view of the array is taken). Please do not try to answer the question in terms of the snippet of code below --- the list I use to index the element is not always short (i.e. thousands of elements, not 4) and the list is a product of an algorithm and hence the number are not necessarily ordered, etc.

For example in the code below columns 1,2 and 3 are selected in both cases but only in the first case a view of the data is returned:

>>> a[:,1:4]
>>> b = a[:,1:4]
>>> b.base is a
True
>>> c = a[:,[1,3,2]]
>>> c.base is a
False
like image 850
methane Avatar asked May 23 '14 01:05

methane


1 Answers

Fancy indexing (using a list of indices to access elements of an array) always produces a copy, as there is no way for numpy to translate it into a new view of the same data, but with a different fixed stride and shape, starting from a particular element.

Under the hood, a numpy array is a pointer to the first element in memory of an array, a dtype, shape and information about how far to move in memory to get to each of the dimensions (next row, column, etc) and some flags. A view on some pre-existing memory just points to some element in that array and fiddles with the stride and shape. Fancy indexing generally specifies random access into that pre-existing memory and you can't force that data into the necessary form, so a copy has to be made.

like image 69
JoshAdel Avatar answered Oct 21 '22 13:10

JoshAdel