Given a byte buffer, dtype, shape and strides, how to create Numpy ndarray

Question

I have a buffer, dtype, shape and strides. I want to create a Numpy ndarray which reuses the memory of the buffer.

There is numpy.frombuffer which creates a 1D array from a buffer and reuses the memory. However, I'm not sure if I can easily and safely reshape it and set the strides.

There is the numpy.ndarray constructor which can refer to a buffer but I'm not sure if it will reuse the memory or if it will copy it (it's not clear from the documentation).

So, will the numpy.ndarray constructor do what I want? Or what can I use instead?

---

Ok, so I'm trying to figure out myself now what the numpy.ndarray constructor is really doing. The code is here. It uses PyArray_BufferConverter to convert the buffer argument. Then it will call PyArray_NewFromDescr_int which can be seen here. If data is passed in there, it will fa->flags &= ~NPY_ARRAY_OWNDATA;.

Answer 1

As mentioned in the comment by @hpaulj, you can accomplish this using the stride_tricks module. You need both np.frombuffer and np.lib.stride_tricks.as_strided:

Gather data from NumPy array

In [1]: import numpy as np
In [2]: x = np.random.random((3, 4)).astype(dtype='f4')
In [3]: buffer = x.data
In [4]: dtype = x.dtype
In [5]: shape = x.shape
In [6]: strides = x.strides

Recreate NumPy array

In [7]: xx = np.frombuffer(buffer, dtype)
In [8]: xx = np.lib.stride_tricks.as_strided(xx, shape, strides)

Verify results

In [9]: x
Out[9]: 
array([[ 0.75343359,  0.20676662,  0.83675659,  0.99904215],
       [ 0.37182721,  0.83846378,  0.6888299 ,  0.57195812],
       [ 0.39905572,  0.7258808 ,  0.88316005,  0.2187883 ]], dtype=float32)

In [10]: xx
Out[10]: 
array([[ 0.75343359,  0.20676662,  0.83675659,  0.99904215],
       [ 0.37182721,  0.83846378,  0.6888299 ,  0.57195812],
       [ 0.39905572,  0.7258808 ,  0.88316005,  0.2187883 ]], dtype=float32)

In [11]: x.strides
Out[11]: (16, 4)
In [12]: xx.strides
Out[12]: (16, 4)