Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Converting numpy arrays of arrays into one whole numpy array

I want to turn my array of array into just a single array. From something like :

array([ array([[0, 0, 0, ..., 1, 0, 0],
       [0, 1, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 2, 0, 0],
       ..., 
       array([[0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 8, 0, 2],
       ..., 
       [0, 0, 0, ..., 0, 0, 0],
       [1, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 1, 0, 0]], dtype=uint8)], dtype=object)

which has size (10,) to just the 3D numpy array which is of size (10,518, 32)

array([[[0, 0, 0, ..., 0, 0, 0],
        [0, 0, 0, ..., 0, 0, 0],
        [0, 0, 0, ..., 0, 0, 0],
        ..., 
        [0, 0, 0, ..., 0, 0, 0],
        [0, 0, 0, ..., 0, 0, 0],
        [0, 0, 0, ..., 0, 0, 0]]], dtype=uint8)

I've tried converting everything into a list then do np.asarray and also tried defining everything as the same dtype=uint8 but I couldn't get it into the 3D form.

like image 900
ROBOTPWNS Avatar asked Feb 04 '16 04:02

ROBOTPWNS


People also ask

How do I combine multiple NumPy arrays into one?

Use numpy. concatenate() to merge the content of two or multiple arrays into a single array. This function takes several arguments along with the NumPy arrays to concatenate and returns a Numpy array ndarray. Note that this method also takes axis as another argument, when not specified it defaults to 0.

How do I combine two arrays in Python?

NumPy's concatenate function can be used to concatenate two arrays either row-wise or column-wise. Concatenate function can take two or more arrays of the same shape and by default it concatenates row-wise i.e. axis=0. The resulting array after row-wise concatenation is of the shape 6 x 3, i.e. 6 rows and 3 columns.

How do I append a NumPy array to a NumPy array?

You can use numpy. append() function to add an element in a NumPy array. You can pass the NumPy array and multiple values as arguments to the append() function. It doesn't modify the existing array but returns a copy of the passed array with given values added.

How do I flatten an array in NumPy?

flatten() function return a copy of the array collapsed into one dimension. Parameters : order : [{'C', 'F', 'A', 'K'}, optional] 'C' means to flatten in row-major (C-style) order. 'F' means to flatten in column-major (Fortran- style) order.


1 Answers

np.concatenate should do the trick:

Make an object array of arrays:

In [23]: arr=np.empty((4,),dtype=object)
In [24]: for i in range(4):arr[i]=np.ones((2,2),int)*i
In [25]: arr
Out[25]: 
array([array([[0, 0],
       [0, 0]]), array([[1, 1],
       [1, 1]]),
       array([[2, 2],
       [2, 2]]), array([[3, 3],
       [3, 3]])], dtype=object)

In [28]: np.concatenate(arr)
Out[28]: 
array([[0, 0],
       [0, 0],
       [1, 1],
       [1, 1],
       [2, 2],
       [2, 2],
       [3, 3],
       [3, 3]])

Or with a reshape:

In [26]: np.concatenate(arr).reshape(4,2,2)
Out[26]: 
array([[[0, 0],
        [0, 0]],

       [[1, 1],
        [1, 1]],

       [[2, 2],
        [2, 2]],

       [[3, 3],
        [3, 3]]])
In [27]: _.shape
Out[27]: (4, 2, 2)

concatenate effectively treats its input as a list of arrays. So it works regardless of whether this is an object array, a list, or 3d array.

This can't be done simply with a reshape. arr is an array of pointers - pointing to arrays located elsewhere in memory. To get a single 3d array, all of the pieces will have to be copied into one buffer. That's what concatenate does - it creates a large empty file, and copies each array, but it does it in compiled code.


np.array does not change it:

In [37]: np.array(arr).shape
Out[37]: (4,)

but treating arr as a list of arrays does work (but is slower than the concatenate version - array analyses its inputs more).

In [38]: np.array([x for x in arr]).shape
Out[38]: (4, 2, 2)
like image 114
hpaulj Avatar answered Sep 22 '22 06:09

hpaulj