Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Convert pandas DateTimeIndex to Unix Time?

Tags:

python

pandas

What is the idiomatic way of converting a pandas DateTimeIndex to (an iterable of) Unix Time? This is probably not the way to go:

[time.mktime(t.timetuple()) for t in my_data_frame.index.to_pydatetime()] 
like image 342
Christian Geier Avatar asked Mar 04 '13 14:03

Christian Geier


People also ask

How do you convert UTC time to UNIX time in Python?

DateTime to Unix timestamp in UTC Timezone In the time module, the timegm function returns a Unix timestamp. The timetuple() function of the datetime class returns the datetime's properties as a named tuple. To obtain the Unix timestamp, use print(UTC).

How do I convert DatetimeIndex to Python?

To convert the index of a DataFrame to DatetimeIndex , use Pandas' to_datetime(~) method.

How do I get the current time in Unix Python?

To get the current epoch/UNIX timestamp in Python: Import the time module. Call the time. time() function.


2 Answers

As DatetimeIndex is ndarray under the hood, you can do the conversion without a comprehension (much faster).

In [1]: import numpy as np  In [2]: import pandas as pd  In [3]: from datetime import datetime  In [4]: dates = [datetime(2012, 5, 1), datetime(2012, 5, 2), datetime(2012, 5, 3)]    ...: index = pd.DatetimeIndex(dates)    ...:  In [5]: index.astype(np.int64) Out[5]: array([1335830400000000000, 1335916800000000000, 1336003200000000000],          dtype=int64)  In [6]: index.astype(np.int64) // 10**9 Out[6]: array([1335830400, 1335916800, 1336003200], dtype=int64)  %timeit [t.value // 10 ** 9 for t in index] 10000 loops, best of 3: 119 us per loop  %timeit index.astype(np.int64) // 10**9 100000 loops, best of 3: 18.4 us per loop 
like image 92
root Avatar answered Sep 24 '22 03:09

root


Note: Timestamp is just unix time with nanoseconds (so divide it by 10**9):

[t.value // 10 ** 9 for t in tsframe.index] 

For example:

In [1]: t = pd.Timestamp('2000-02-11 00:00:00')  In [2]: t Out[2]: <Timestamp: 2000-02-11 00:00:00>  In [3]: t.value Out[3]: 950227200000000000L  In [4]: time.mktime(t.timetuple()) Out[4]: 950227200.0 

As @root points out it's faster to extract the array of values directly:

tsframe.index.astype(np.int64) // 10 ** 9 
like image 38
Andy Hayden Avatar answered Sep 24 '22 03:09

Andy Hayden