Please, help me. I want to resample based on 1D. I have following format of data. I want to use resampling in pandas.
I want to resample based on Date and product and also fill the missing values.
But I keep getting this mistake: I tried like 5 options and mistake only changes after "instance of": I saw there Multiindex, Index.
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'RangeIndex'
product value date
A 1.52 2016-01-01
A NULL 2016-09-20
A 1.33 2018-08-02
B 1.30 2016-01-01
B NULL 2017-01-02
B 1.54 2017-03-10
B 2.08 2017-06-28
B 2.33 2018-08-02
I put these data into
df.reset_index().set_index('date','sku')
df= df.groupby('product').resample('1D')['value'].ffill().bfill().ffill()
I tried also:
df = df.set_index(['date','sku'])
df = df.set_index('date','sku')
df = df.reset_index().set_index(['date','sku'])
Please, can you explain me what I am doing wrong? Thanks!
Today morning it was working on these data and the command from Jezrael:
df = df.set_index('date').groupby('product').resample('1D')['value'].ffill()
product value date
0 A 1.52 2016-01-01
1 A NaN 2016-09-20
2 A 1.87 2018-08-02
3 B 2.33 2016-01-01
4 B NaN 2016-09-20
5 B 4.55 2018-08-02
But suddenly it doesnt anymore. Now I have Index in the error statement.
You need DatetimeIndex if working with DataFrameGroupBy.resample, also bfill is omited because if some only NaNs groups is possible these data are replaced from another groups:
#if necessary convert to datetimes
#df['date'] = pd.to_datetime(df['date'])
df = df.set_index('date').groupby('product').resample('1D')['value'].ffill()
print (df)
product date
A 2016-01-01 1.52
2016-01-02 1.52
2016-01-03 1.52
2016-01-04 1.52
2016-01-05 1.52
2016-01-06 1.52
2016-01-07 1.52
2016-01-08 1.52
2016-01-09 1.52
2016-01-10 1.52
2016-01-11 1.52
2016-01-12 1.52
Changed sample for better explanation:
print (df)
product value date
0 A 1.52 2016-01-01
1 A NaN 2016-01-03
2 B NaN 2017-01-02
3 B NaN 2017-01-03
4 C 1.54 2017-03-10
5 C 2.08 2017-03-12
6 C 2.33 2017-03-14
df1 = df.set_index('date').groupby('product').resample('1D')['value'].ffill()
print (df1)
product date
A 2016-01-01 1.52
2016-01-02 1.52
2016-01-03 NaN < NaN is not changed because in original data
B 2017-01-02 NaN <- only NaN group B
2017-01-03 NaN
C 2017-03-10 1.54
2017-03-11 1.54
2017-03-12 2.08
2017-03-13 2.08
2017-03-14 2.33
Name: value, dtype: float64
df11 = df.set_index('date').groupby('product').resample('1D')['value'].ffill().bfill()
print (df11)
product date
A 2016-01-01 1.52
2016-01-02 1.52
2016-01-03 1.54 <- back filling value from group C
B 2017-01-02 1.54 <- back filling value from group C
2017-01-03 1.54 <- back filling value from group C
C 2017-03-10 1.54
2017-03-11 1.54
2017-03-12 2.08
2017-03-13 2.08
2017-03-14 2.33
Name: value, dtype: float64
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With