I have a Pandas dataframe that I want to forward fill HORIZONTALLY but I don't want to forward fill past the last entry in each row. This is time series pricing data on products where some have been discontinued so I dont want the last value recorded to be forward filled to current.
FWDFILL.apply(lambda series: series.iloc[:,series.last_valid_index()].ffill(axis=1))
^The code I have included does what I want but it does it VERTICALLY. This could maybe help people as a starting point.
>>> print(FWDFILL)
1 1 NaN NaN 2 NaN
2 NaN 1 NaN 5 NaN
3 NaN 3 1 NaN NaN
4 NaN NaN NaN NaN NaN
5 NaN 5 NaN NaN 1
Desired Output:
1 1 1 1 2 NaN
2 NaN 1 1 5 NaN
3 NaN 3 1 NaN NaN
4 NaN NaN NaN NaN NaN
5 NaN 5 5 5 1
IIUC, you need to apply
with axis=1
, so you are applying to dataframe rows instead of dataframe columns.
df.apply(lambda x: x[:x.last_valid_index()].ffill(), axis=1)
Output:
1 2 3 4 5
0
1 1.0 1.0 1.0 2.0 NaN
2 NaN 1.0 1.0 5.0 NaN
3 NaN 3.0 1.0 NaN NaN
4 NaN NaN NaN NaN NaN
5 NaN 5.0 5.0 5.0 1.0
Usage of bfill
and ffill
s1=df.ffill(1)
s2=df.bfill(1)
df=df.mask(s1.notnull()&s2.notnull(),s1)
df
Out[222]:
1 2 3 4 5
1 1.0 1.0 1.0 2.0 NaN
2 NaN 1.0 1.0 5.0 NaN
3 NaN 3.0 1.0 NaN NaN
4 NaN NaN NaN NaN NaN
5 NaN 5.0 5.0 5.0 1.0
Or just using interpolate
df.mask(df.interpolate(axis=1,limit_area='inside').notnull(),df.ffill(1))
Out[226]:
1 2 3 4 5
1 1.0 1.0 1.0 2.0 NaN
2 NaN 1.0 1.0 5.0 NaN
3 NaN 3.0 1.0 NaN NaN
4 NaN NaN NaN NaN NaN
5 NaN 5.0 5.0 5.0 1.0
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With