Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas fillna on a multiindex series

Tags:

python

pandas

The series_A with NAs is indexed by a MultiIndex (X, Y), while the values to fill in are in the Series_B, which is indexed by X. How to effectively approach this kind of problem?

For example, here is the series_A:

bar  one    0.299368
     two         NaN
baz  one   -0.863838
     two   -0.251905
foo  one    1.063327
     two         NaN
qux  one    0.206053
     two    0.408204

and series_B contains values to fill in:

bar  0.123
foo  0.456
like image 788
James Wong Avatar asked Feb 04 '23 08:02

James Wong


1 Answers

Method 1
use unstack to put first level of index into columns and then use fillna

series_A.unstack(0).fillna(series_B).unstack().dropna()

bar  one    0.299368
     two    0.123000
baz  one   -0.863838
     two   -0.251905
foo  one    1.063327
     two    0.456000
qux  one    0.206053
quz  two    0.408204
dtype: float64

Method 2
use add method to leverage its level and fill_value parameters then combine_first

series_A.combine_first(series_A.add(series_B, level=0, fill_value=0))

bar  one    0.299368
     two    0.123000
baz  one   -0.863838
     two   -0.251905
foo  one    1.063327
     two    0.456000
qux  one    0.206053
quz  two    0.408204
dtype: float64

Method 3
use map on the Index object returned from series_A.index.get_level_vaues(0) with the callable series_B.get

series_A.fillna(
    pd.Series(series_A.index.get_level_values(0).map(series_B.get), series_A.index)
)

bar  one    0.299368
     two    0.123000
baz  one   -0.863838
     two   -0.251905
foo  one    1.063327
     two    0.456000
qux  one    0.206053
quz  two    0.408204
dtype: float64

Method 4
use np.isnan and np.flatnonzero to find the positions of where the np.nans are. Then find the values to insert with get_level_values and map. Finally, place into location with iloc

i = np.flatnonzero(np.isnan(series_A.values))
series_A.iloc[i] = series_A.index.get_level_values(0)[i].map(series_B.get)
series_A

bar  one    0.299368
     two    0.123000
baz  one   -0.863838
     two   -0.251905
foo  one    1.063327
     two    0.456000
qux  one    0.206053
quz  two    0.408204
dtype: float64
like image 87
piRSquared Avatar answered Feb 15 '23 02:02

piRSquared