numpy stride_tricks.as_strided vs list comprehension for rolling window

Question

When dealing with rolling windows, I wrote my functions in the way like list comprehension

[np.std(x[i:i+framesize]) for i in range(0, len(x)-framesize, hopsize)])]

Recently I discovered numpy.lib.stride_tricks.as_strided and found it is used widely for rolling windows (for example, this post), even though it is a "hidden" function.

In this issue concerning why stride_tricks.as_strided is undocumented, it's mentioned that

Intentionally! It's dangerous! It was just low-level plumbing to help implement broadcast_arrays().

Is there any advantage for stride_tricks.as_strided over the list comprehension or a for loop? I had a look at the source code of stride_tricks but gained little.

Answer 1

From this post, we can use strided_app to basically get sliding views into the array and it also allows us to specify the hopsize/stepsize. Then, we simply use np.std along the second axis for the final output, like so -

np.std(strided_app(x, framesize, hopsize), axis=1)

Sample run for verification -

In [162]: x = np.random.randint(0,9,(11))

In [163]: framesize = 5

In [164]: hopsize = 3

In [165]: np.array([np.std(x[i:i+framesize]) \
            for i in range(0, len(x)-framesize+1, hopsize)])
Out[165]: array([ 1.62480768,  2.05912603,  1.78885438])

In [166]: np.std(strided_app(x, framesize, hopsize), axis=1)
Out[166]: array([ 1.62480768,  2.05912603,  1.78885438])

Being a view into the input array, these strided operations must be really efficient. Let's find it out!

Runtime test

Loopy approach -

def loopy_app(x, framesize, hopsize):
    return [np.std(x[i:i+framesize]) \
        for i in range(0, len(x)-framesize+1, hopsize)]

Timings -

In [185]: x = np.random.randint(0,9,(1001))

In [186]: framesize = 5

In [187]: hopsize = 3

In [188]: %timeit loopy_app(x, framesize, hopsize)
10 loops, best of 3: 17.8 ms per loop

In [189]: %timeit np.std(strided_app(x, framesize, hopsize), axis=1)
10000 loops, best of 3: 111 µs per loop

So, to answer the question on efficiency with strides, the timings should help prove a point there!