Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Non-overlapping rolling windows in pandas dataframes

I am familiar with the Pandas Rolling window functions, but they always have a step size of 1. I want to do a moving aggregate function in Pandas, but where the entries don't overlap.

In this Dataframe: enter image description here

df.rolling(2).min()

will yield:

N/A 519 566 727 1099 12385

But I want a fixed window with a step size of 2, so it yields:

519 727 12385

Because with a fixed window, it should step over by the size of that window instead.

like image 942
sparkonhdfs Avatar asked Aug 21 '19 16:08

sparkonhdfs


1 Answers

There's no such built in argument in the rolling function, but you can compute the usual rolling function and then skip every nth row (where n=2 in your case).

df.rolling(n).min()[n-1::n]

As you mentioned in your comment, this might result in many redudant computations which will be ignored (especially if n is large). Instead, you could use the following code which partitions (groups) the data into bins of size n:

df.groupby(df.index // n).min()

I did not check if it's indeed more efficient, but I believe it should be.

like image 68
Itay Avatar answered Sep 24 '22 01:09

Itay