Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python pandas convert datetime to timestamp effectively through dt accessor

People also ask

What is DT in datetime Python?

dt can be used to access the values of the series as datetimelike and return several properties. Pandas Series. dt. date attribute return a numpy array of python datetime.

What is DatetimeIndex pandas?

class pandas. DatetimeIndex [source] Immutable ndarray of datetime64 data, represented internally as int64, and which can be boxed to Timestamp objects that are subclasses of datetime and carry metadata such as frequency information.

What is Strftime in pandas?

dt. strftime() function is used to convert to Index using specified date_format. The function return an Index of formatted strings specified by date_format, which supports the same string format as the python standard library.


I think you need convert first to numpy array by values and cast to int64 - output is in ns, so need divide by 10 ** 9:

df['ts'] = df.datetime.values.astype(np.int64) // 10 ** 9
print (df)
              datetime          ts
0  2016-01-01 00:00:01  1451606401
1  2016-01-01 01:00:01  1451610001
2  2016-01-01 02:00:01  1451613601
3  2016-01-01 03:00:01  1451617201
4  2016-01-01 04:00:01  1451620801
5  2016-01-01 05:00:01  1451624401
6  2016-01-01 06:00:01  1451628001
7  2016-01-01 07:00:01  1451631601
8  2016-01-01 08:00:01  1451635201
9  2016-01-01 09:00:01  1451638801
10 2016-01-01 10:00:01  1451642401
11 2016-01-01 11:00:01  1451646001
12 2016-01-01 12:00:01  1451649601
13 2016-01-01 13:00:01  1451653201
14 2016-01-01 14:00:01  1451656801
15 2016-01-01 15:00:01  1451660401
16 2016-01-01 16:00:01  1451664001
17 2016-01-01 17:00:01  1451667601
18 2016-01-01 18:00:01  1451671201
19 2016-01-01 19:00:01  1451674801
20 2016-01-01 20:00:01  1451678401
21 2016-01-01 21:00:01  1451682001
22 2016-01-01 22:00:01  1451685601
23 2016-01-01 23:00:01  1451689201
24 2016-01-02 00:00:01  1451692801

to_timestamp is used for converting from period to datetime index.


I think you should not use apply, simply astype would be fine:

df['ts'] = df.datetime.astype('int64') // 10**9

There's also another method to do this using the "hidden" attribute of DatetimeIndex called asi8, which creates an integer timestamp.

pd.DatetimeIndex(df.datetime).asi8

Wes McKinney suggested it in this tangentially related stackoverflow question linked here


If you don't want to use numpy you can use pure pandas conversions

df['ts'] = pd.to_timedelta(df['datetime'], unit='ns').dt.total_seconds().astype(int)

One option would be to use a lambda expressions like such

df['datetime'] = df['datetime'].apply(lambda x: pd.Timestamp(x))