Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas replace NaN with NaT

Tags:

python

pandas

I have a series with some NaNs that I need to replace with NaTs. How can I do this?

Here's a simple example with what I've tried so far:

>>> s = pd.Series([np.NaN, np.NaN])
>>> s.fillna(pd.NaT)
0   NaN
1   NaN
dtype: float64
>>> s.replace(np.NaN, pd.NaT)
0   NaN
1   NaN
dtype: float64
>>> s.where(pd.notnull(s), pd.NaT)
0    NaN
1    NaN
dtype: object

pandas version: 0.16.2

numpy version: 1.9.2

python version: 2.7.10

like image 890
LateCoder Avatar asked May 11 '16 15:05

LateCoder


1 Answers

Convert the dtype first as NaT is meaningless when the dtype is float which is the dtype initially:

In [90]:
s.astype(np.datetime64).fillna(pd.NaT)

Out[90]:
0   NaT
1   NaT
dtype: datetime64[ns]

if You have non-NaN values in the Series then use to_datetime:

In [97]:
s = pd.Series([np.NaN, np.NaN, 1.0])
pd.to_datetime(s)

Out[97]:
0                             NaT
1                             NaT
2   1970-01-01 00:00:00.000000001
dtype: datetime64[ns]
like image 82
EdChum Avatar answered Nov 01 '22 18:11

EdChum