First column name with non null value by row pandas

Question

I want know the first year with incoming revenue for various projects.

Given the following, dataframe:

ID  Y1      Y2      Y3
0   NaN     8       4
1   NaN     NaN     1
2   NaN     NaN     NaN
3   5       3       NaN

I would like to return the name of the first column with a non-null value by row.

In this case, I would want to return:

['Y2','Y3',NaN,'Y1']

My goal is to add this as a column to the original dataframe.

The following code mostly works, but is really clunky.

import pandas as pd
import numpy as np

df = pd.DataFrame({'Y1':[np.nan, np.nan, np.nan, 5],'Y2':[8, np.nan, np.nan, 3], 'Y3':[4, 1, np.nan, np.nan]})
df['first'] = np.nan

for ID in df.index:
row = df.loc[ID,]
for i in range(0,len(row)):
    if (~pd.isnull(row[i])):
        df.loc[ID,'first'] = row.index[i]
        break

returns:

   Y1  Y2  Y3  first
0 NaN  8   4   Y2   
1 NaN NaN  1   Y3   
2 NaN NaN NaN  first
3  5   3  NaN  Y1   

Does anyone know a more elegant solution?

Answer 1

You can apply first_valid_index to each row in the dataframe using a lambda expression with axis=1 to specify rows.

>>> df.apply(lambda row: row.first_valid_index(), axis=1)
ID
0      Y2
1      Y3
2    None
3      Y1
dtype: object

To apply it to your dataframe:

df = df.assign(first = df.apply(lambda row: row.first_valid_index(), axis=1))

>>> df
    Y1  Y2  Y3 first
ID                  
0  NaN   8   4    Y2
1  NaN NaN   1    Y3
2  NaN NaN NaN  None
3    5   3 NaN    Y1