Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Replace Nulls in DataFrame with Max in Row

Is there a way (more efficient than using a for loop) to replace all the nulls in a Pandas' DataFrame with the max value in its respective row.

like image 990
rhaskett Avatar asked Jul 29 '15 16:07

rhaskett


1 Answers

I guess that is what you are looking for:

import pandas as pd  

df = pd.DataFrame({'a': [1, 2, 0], 'b': [3, 0, 10], 'c':[0, 5, 34]})


   a   b   c
0  1   3   0
1  2   0   5
2  0  10  34

You can use apply, iterate over all rows and replace 0 by the maximal number of the row by using the replace function which gives you the expected output:

df.apply(lambda row: row.replace(0, max(row)), axis=1)

    a   b   c
0   1   3   3
1   2   5   5
2  34  10  34

If you want to to replace NaN - which seemed to be your actual goal according to your comment - you can use

df = pd.DataFrame({'a': [1, 2, np.nan], 'b': [3, np.nan, 10], 'c':[np.nan, 5, 34]})

     a     b     c
0  1.0   3.0   NaN
1  2.0   NaN   5.0
2  NaN  10.0  34.0

df.T.fillna(df.max(axis=1)).T

yielding

      a     b     c
0   1.0   3.0   3.0
1   2.0   5.0   5.0
2  34.0  10.0  34.0

which might be more efficient (have not done the timings) than

df.apply(lambda row: row.fillna(row.max()), axis=1)

Please note that

df.apply(lambda row: row.fillna(max(row)), axis=1)

does not work in each case as explained here.

like image 94
Cleb Avatar answered Oct 11 '22 01:10

Cleb