Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ValueError: index must be monotonic when applying rolling("2H").mean()

Tags:

python

pandas

I have the following DataFrame df:

                   TIME     DELAY
0   2016-01-01 06:30:00     0
1   2016-01-01 14:10:00     2
2   2016-01-01 07:05:00     2
3   2016-01-01 11:00:00     1
4   2016-01-01 10:40:00     0
5   2016-01-01 08:10:00     7
6   2016-01-01 11:35:00     2
7   2016-01-02 13:50:00     2
8   2016-01-02 14:50:00     4
9   2016-01-02 14:05:00     1

Please notice that row are not sorted by a datetime object.

For each row I want to know the average delay for the last 2 hours. To do this task, I executed the following code:

df.index = pd.DatetimeIndex(df["TIME"])
df["DELAY_LAST2HOURS"] = df["DELAY"].rolling("2H").mean()

However I got this error:

ValueError: index must be monotonic

How can I properly solve my task?

like image 795
ScalaBoy Avatar asked Jan 29 '19 13:01

ScalaBoy


1 Answers

Problem is DatetimeIndex is not sorted, so need DataFrame.sort_index:

df.index = pd.DatetimeIndex(df["TIME"])
df = df.sort_index()
df["DELAY_LAST2HOURS"] = df["DELAY"].rolling("2H").mean()
print (df)
                                    TIME  DELAY  DELAY_LAST2HOURS
TIME                                                             
2016-01-01 06:30:00  2016-01-01 06:30:00      0          0.000000
2016-01-01 07:05:00  2016-01-01 07:05:00      2          1.000000
2016-01-01 08:10:00  2016-01-01 08:10:00      7          3.000000
2016-01-01 10:40:00  2016-01-01 10:40:00      0          0.000000
2016-01-01 11:00:00  2016-01-01 11:00:00      1          0.500000
2016-01-01 11:35:00  2016-01-01 11:35:00      2          1.000000
2016-01-01 14:10:00  2016-01-01 14:10:00      2          2.000000
2016-01-02 13:50:00  2016-01-02 13:50:00      2          2.000000
2016-01-02 14:05:00  2016-01-02 14:05:00      1          1.500000
2016-01-02 14:50:00  2016-01-02 14:50:00      4          2.333333

All together should be if not necessary original TIME column:

df["TIME"] = pd.to_datetime(df["TIME"])

df = df.set_index('TIME').sort_index()
df["DELAY_LAST2HOURS"] = df["DELAY"].rolling("2H").mean()
print (df)
                     DELAY  DELAY_LAST2HOURS
TIME                                        
2016-01-01 06:30:00      0          0.000000
2016-01-01 07:05:00      2          1.000000
2016-01-01 08:10:00      7          3.000000
2016-01-01 10:40:00      0          0.000000
2016-01-01 11:00:00      1          0.500000
2016-01-01 11:35:00      2          1.000000
2016-01-01 14:10:00      2          2.000000
2016-01-02 13:50:00      2          2.000000
2016-01-02 14:05:00      1          1.500000
2016-01-02 14:50:00      4          2.333333

EDIT:

df["TIME"] = pd.to_datetime(df["TIME"])
df = df.sort_values('TIME').set_index('TIME')

df["DELAY_LAST2HOURS"] = df["DELAY"].rolling("2H").mean()
like image 172
jezrael Avatar answered Nov 02 '22 01:11

jezrael