How can I implement x[i][j] = y[i+j] efficiently in numpy?

Question

Let x be a matrix with a shape of (A,B) and y be an array with a size of A+B-1.

for i in range(A):
    for j in range(B):
        x[i][j] = y[i+j]

How can I implement equivalent code efficiently using functions in numpy?

Answer 1

Approach #1 Using Scipy's hankel -

from scipy.linalg import hankel

x = hankel(y[:A],y[A-1:]

Approach #2 Using NumPy broadcasting -

x = y[np.arange(A)[:,None] + np.arange(B)]

Approach #3 Using NumPy strides technique -

n = y.strides[0]
x = np.lib.stride_tricks.as_strided(y, shape=(A,B), strides=(n,n))

---

Runtime test -

In [93]: def original_app(y,A,B):
    ...:     x = np.zeros((A,B))
    ...:     for i in range(A):
    ...:         for j in range(B):
    ...:             x[i][j] = y[i+j]
    ...:     return x
    ...: 
    ...: def strided_method(y,A,B):
    ...:     n = y.strides[0]
    ...:     return np.lib.stride_tricks.as_strided(y, shape=(A,B), strides=(n,n))
    ...: 

In [94]: # Inputs
    ...: A,B = 100,100
    ...: y = np.random.rand(A+B-1)
    ...: 

In [95]: np.allclose(original_app(y,A,B),hankel(y[:A],y[A-1:]))
Out[95]: True

In [96]: np.allclose(original_app(y,A,B),y[np.arange(A)[:,None] + np.arange(B)])
Out[96]: True

In [97]: np.allclose(original_app(y,A,B),strided_method(y,A,B))
Out[97]: True

In [98]: %timeit original_app(y,A,B)
100 loops, best of 3: 5.29 ms per loop

In [99]: %timeit hankel(y[:A],y[A-1:])
10000 loops, best of 3: 114 µs per loop

In [100]: %timeit y[np.arange(A)[:,None] + np.arange(B)]
10000 loops, best of 3: 60.5 µs per loop

In [101]: %timeit strided_method(y,A,B)
10000 loops, best of 3: 22.4 µs per loop

Additional ways based on strides -

It seems strides technique has been used at few places : extract_patches and view_as_windows that are being used in such image-processing based modules. So, with those, we have two more approaches -

from skimage.util.shape import view_as_windows
from sklearn.feature_extraction.image import extract_patches

x = extract_patches(y,(B))
x = view_as_windows(y,(B))

In [151]: np.allclose(original_app(y,A,B),extract_patches(y,(B)))
Out[151]: True

In [152]: np.allclose(original_app(y,A,B),view_as_windows(y,(B)))
Out[152]: True

In [153]: %timeit extract_patches(y,(B))
10000 loops, best of 3: 62.4 µs per loop

In [154]: %timeit view_as_windows(y,(B))
10000 loops, best of 3: 108 µs per loop