I have a dataframe:
df = pd.DataFrame(np.random.randint(0,100,size=(5, 2)), columns=list('AB'))
A B
0 92 65
1 61 97
2 17 39
3 70 47
4 56 6
Here are 5% quantiles:
down_quantiles = df.quantile(0.05)
A 24.8
B 12.6
And here is the mask for values that are lower than quantiles:
outliers_low = (df < down_quantiles)
A B
0 False False
1 False False
2 True False
3 False False
4 False True
I want to set values in df
lower than quantile to its column quantile. I can do it like this:
df[outliers_low] = np.nan
df.fillna(down_quantiles, inplace=True)
A B
0 92.0 65.0
1 61.0 97.0
2 24.8 39.0
3 70.0 47.0
4 56.0 12.6
But certainly there should be a more elegant way. How can I do this without fillna
?
Thanks.
You can use DF.mask()
method. Wherever there is a presence of a True
instance, the values from the other series get's replaced aligned as per matching column names by providing axis=1
.
df.mask(outliers_low, down_quantiles, axis=1)
Another variant would be to use DF.where()
method after inverting your boolean mask using the tilde (~
) symbol.
df.where(~outliers_low, down_quantiles, axis=1)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With