Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas fillna on datetime object

Tags:

python

pandas

I'm trying to run fillna on a column of type datetime64[ns]. When I run something like: df['date'].fillna(datetime("2000-01-01"))

I get: TypeError: an integer is required

Any way around this?

like image 416
Luke Avatar asked Jan 15 '14 18:01

Luke


2 Answers

This should work in 0.12 and 0.13 (just released).

@DSM points out that datetimes are constructed like: datetime.datetime(2012,1,1) SO the error is from failing to construct the value the you are passing to fillna. Note that using a Timestamp WILL parse the string.

In [3]: s = Series(date_range('20130101',periods=10))

In [4]: s.iloc[3] = pd.NaT

In [5]: s.iloc[7] = pd.NaT

In [6]: s
Out[6]: 
0   2013-01-01 00:00:00
1   2013-01-02 00:00:00
2   2013-01-03 00:00:00
3                   NaT
4   2013-01-05 00:00:00
5   2013-01-06 00:00:00
6   2013-01-07 00:00:00
7                   NaT
8   2013-01-09 00:00:00
9   2013-01-10 00:00:00
dtype: datetime64[ns]

datetime.datetime will work as well

In [7]: s.fillna(Timestamp('20120101'))
Out[7]: 
0   2013-01-01 00:00:00
1   2013-01-02 00:00:00
2   2013-01-03 00:00:00
3   2012-01-01 00:00:00
4   2013-01-05 00:00:00
5   2013-01-06 00:00:00
6   2013-01-07 00:00:00
7   2012-01-01 00:00:00
8   2013-01-09 00:00:00
9   2013-01-10 00:00:00
dtype: datetime64[ns]
like image 113
Jeff Avatar answered Sep 27 '22 19:09

Jeff


This example is works with dynamic data if you want to replace NaT data in rows with data from another DateTime data.

df['column_with_NaT'].fillna(df['dt_column_with_thesame_index'], inplace=True)

It's works for me when I was updated some rows in DateTime column and not updated rows had NaT value, and I've been needed to inherit old series data. And this code above resolve my problem. Sry for the not perfect English )

like image 29
nlavr Avatar answered Sep 27 '22 20:09

nlavr