Menu
My dataframe contains both NaT and NaN values
Date/Time_entry Entry Date/Time_exit Exit
0 2015-11-11 10:52:00 19.9900 2015-11-11 11:30:00 20.350
1 2015-11-11 11:36:00 20.4300 2015-11-11 11:38:00 20.565
2 2015-11-11 11:44:00 21.0000 NaT NaN
3 2009-04-20 10:28:00 13.7788 2009-04-20 10:46:00 13.700
I want to fill NaT with dates and NaN with numbers. Fillna(4) method replaces both NaT and NaN with 4. Is it possible to differentiate between NaT and NaN somehow?
My current workaround is to df[column].fillna()
550
Another bizarre thing about missing values in Pandas is that some varieties are equal to themselves and others aren't. NaN doesn't equal NaN . And NaT doesn't equal NaT . But None does equal None .
We can replace the NaN with an empty string using df. replace() function. This function will replace an empty string inplace of the NaN value.
Since NaTs pertain to datetime columns, you can exclude them when applying your filling operation.
u = df.select_dtypes(exclude=['datetime'])
df[u.columns] = u.fillna(4)
df
Date/Time_entry Entry Date/Time_exit Exit
0 2015-11-11 10:52:00 19.9900 2015-11-11 11:30:00 20.350
1 2015-11-11 11:36:00 20.4300 2015-11-11 11:38:00 20.565
2 2015-11-11 11:44:00 21.0000 NaT 4.000
3 2009-04-20 10:28:00 13.7788 2009-04-20 10:46:00 13.700
Similarly, to fill NaT values only, change "exclude" to "include" in the code above.
u = df.select_dtypes(include=['datetime'])
df[u.columns] = u.fillna(pd.to_datetime('today'))
df
Date/Time_entry Entry Date/Time_exit Exit
0 2015-11-11 10:52:00 19.9900 2015-11-11 11:30:00.000000 20.350
1 2015-11-11 11:36:00 20.4300 2015-11-11 11:38:00.000000 20.565
2 2015-11-11 11:44:00 21.0000 2019-02-17 16:11:09.407466 4.000
3 2009-04-20 10:28:00 13.7788 2009-04-20 10:46:00.000000 13.700
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
