Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: reduce precision pandas timestamp dataframe

Hello I have the following dataframe

df =          Record_ID       Time         94704   2014-03-10 07:19:19.647342         94705   2014-03-10 07:21:44.479363         94706   2014-03-10 07:21:45.479581         94707   2014-03-10 07:21:54.481588         94708   2014-03-10 07:21:55.481804 

Is it possible to the have following?

df1 =          Record_ID       Time         94704   2014-03-10 07:19:19         94705   2014-03-10 07:21:44         94706   2014-03-10 07:21:45         94707   2014-03-10 07:21:54         94708   2014-03-10 07:21:55 
like image 999
emax Avatar asked Sep 28 '15 15:09

emax


People also ask

How do I change timestamp on pandas?

replace() function is used to replace the member values of the given Timestamp. The function implements datetime. replace, and it also handles nanoseconds. Example #1: Use Timestamp.

What is PD timestamp?

Timestamp is the pandas equivalent of python's Datetime and is interchangeable with it in most cases. It's the type used for the entries that make up a DatetimeIndex, and other timeseries oriented data structures in pandas.

What is DatetimeIndex pandas?

class pandas. DatetimeIndex [source] Immutable ndarray of datetime64 data, represented internally as int64, and which can be boxed to Timestamp objects that are subclasses of datetime and carry metadata such as frequency information.


2 Answers

You could convert the underlying datetime64[ns] values to datetime64[s] values using astype:

In [11]: df['Time'] = df['Time'].astype('datetime64[s]')  In [12]: df Out[12]:     Record_ID                Time 0      94704 2014-03-10 07:19:19 1      94705 2014-03-10 07:21:44 2      94706 2014-03-10 07:21:45 3      94707 2014-03-10 07:21:54 4      94708 2014-03-10 07:21:55 

Note that since Pandas Series and DataFrames store all datetime values as datetime64[ns] these datetime64[s] values are automatically converted back to datetime64[ns], so the end result is still stored as datetime64[ns] values, but the call to astype causes the fractional part of the seconds to be removed.

If you wish to have a NumPy array of datetime64[s] values, you could use df['Time'].values.astype('datetime64[s]').

like image 87
unutbu Avatar answered Oct 08 '22 16:10

unutbu


If you really must remove the microsecond part of the datetime, you can use the Timestamp.replace method along with Series.apply method to apply it across the series , to replace the microsecond part with 0. Example -

df['Time'] = df['Time'].apply(lambda x: x.replace(microsecond=0)) 

Demo -

In [25]: df Out[25]:    Record_ID                       Time 0      94704 2014-03-10 07:19:19.647342 1      94705 2014-03-10 07:21:44.479363 2      94706 2014-03-10 07:21:45.479581 3      94707 2014-03-10 07:21:54.481588 4      94708 2014-03-10 07:21:55.481804  In [26]: type(df['Time'][0]) Out[26]: pandas.tslib.Timestamp  In [27]: df['Time'] = df['Time'].apply(lambda x: x.replace(microsecond=0))  In [28]: df Out[28]:    Record_ID                Time 0      94704 2014-03-10 07:19:19 1      94705 2014-03-10 07:21:44 2      94706 2014-03-10 07:21:45 3      94707 2014-03-10 07:21:54 4      94708 2014-03-10 07:21:55 
like image 45
Anand S Kumar Avatar answered Oct 08 '22 16:10

Anand S Kumar