Please, help me. I want to resample based on 1D. I have following format of data. I want to use resampling in pandas.
I want to resample based on Date and product and also fill the missing values.
But I keep getting this mistake: I tried like 5 options and mistake only changes after "instance of": I saw there Multiindex, Index.
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'RangeIndex'
product value date
A 1.52 2016-01-01
A NULL 2016-09-20
A 1.33 2018-08-02
B 1.30 2016-01-01
B NULL 2017-01-02
B 1.54 2017-03-10
B 2.08 2017-06-28
B 2.33 2018-08-02
I put these data into
df.reset_index().set_index('date','sku')
df= df.groupby('product').resample('1D')['value'].ffill().bfill().ffill()
I tried also:
df = df.set_index(['date','sku'])
df = df.set_index('date','sku')
df = df.reset_index().set_index(['date','sku'])
Please, can you explain me what I am doing wrong? Thanks!
Today morning it was working on these data and the command from Jezrael:
df = df.set_index('date').groupby('product').resample('1D')['value'].ffill()
product value date
0 A 1.52 2016-01-01
1 A NaN 2016-09-20
2 A 1.87 2018-08-02
3 B 2.33 2016-01-01
4 B NaN 2016-09-20
5 B 4.55 2018-08-02
But suddenly it doesnt anymore. Now I have Index in the error statement.
You need DatetimeIndex
if working with DataFrameGroupBy.resample
, also bfill
is omited because if some only NaN
s groups is possible these data are replaced from another groups:
#if necessary convert to datetimes
#df['date'] = pd.to_datetime(df['date'])
df = df.set_index('date').groupby('product').resample('1D')['value'].ffill()
print (df)
product date
A 2016-01-01 1.52
2016-01-02 1.52
2016-01-03 1.52
2016-01-04 1.52
2016-01-05 1.52
2016-01-06 1.52
2016-01-07 1.52
2016-01-08 1.52
2016-01-09 1.52
2016-01-10 1.52
2016-01-11 1.52
2016-01-12 1.52
Changed sample for better explanation:
print (df)
product value date
0 A 1.52 2016-01-01
1 A NaN 2016-01-03
2 B NaN 2017-01-02
3 B NaN 2017-01-03
4 C 1.54 2017-03-10
5 C 2.08 2017-03-12
6 C 2.33 2017-03-14
df1 = df.set_index('date').groupby('product').resample('1D')['value'].ffill()
print (df1)
product date
A 2016-01-01 1.52
2016-01-02 1.52
2016-01-03 NaN < NaN is not changed because in original data
B 2017-01-02 NaN <- only NaN group B
2017-01-03 NaN
C 2017-03-10 1.54
2017-03-11 1.54
2017-03-12 2.08
2017-03-13 2.08
2017-03-14 2.33
Name: value, dtype: float64
df11 = df.set_index('date').groupby('product').resample('1D')['value'].ffill().bfill()
print (df11)
product date
A 2016-01-01 1.52
2016-01-02 1.52
2016-01-03 1.54 <- back filling value from group C
B 2017-01-02 1.54 <- back filling value from group C
2017-01-03 1.54 <- back filling value from group C
C 2017-03-10 1.54
2017-03-11 1.54
2017-03-12 2.08
2017-03-13 2.08
2017-03-14 2.33
Name: value, dtype: float64
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With