Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to check if a pandas Series contains Timestamps?

Tags:

python

pandas

Following seems to work at least in some cases:

series.dtype == np.dtype('<M8[ns]')

but it doesn't look pretty and I'm not sure if it works always (with all(?) kinds of Timestamps).

Is there a nicer way for testing whether a column contains Timestamps?

like image 884
Aivar Avatar asked Jul 08 '17 16:07

Aivar


People also ask

How do you count occurrences in pandas series?

Using the size() or count() method with pandas. DataFrame. groupby() will generate the count of a number of occurrences of data present in a particular column of the dataframe.

How do I compare timestamps in pandas?

Comparison between pandas timestamp objects is carried out using simple comparison operators: >, <,==,< = , >=. The difference can be calculated using a simple '–' operator. Given time can be converted to pandas timestamp using pandas. Timestamp() method.

What is timeseries in pandas?

Pandas Time Series Data Structures As mentioned before, it is essentially a replacement for Python's native datetime , but is based on the more efficient numpy. datetime64 data type. The associated Index structure is DatetimeIndex . For time Periods, Pandas provides the Period type.


2 Answers

try this it should work:

pd.core.dtypes.common.is_datetime_or_timedelta_dtype(series)

if you go through pd.core.dtypes.common.is_ you will find many options for checking timestamps. If you want to combine them then you can use logical operators like:

pd.core.dtypes.common.is_datetime64_ns_dtype(ser)|pd.core.dtypes.common.is_timedelta64_ns_dtype(ser)
like image 77
shivsn Avatar answered Sep 27 '22 15:09

shivsn


The approach in @shivsn's answer is simpler and most likely better. This is a harder way that nonetheless may be informative.

See here for more details on dtypes. But in a nutshell, the second character of the dtype string should be 'M' for a Datetime and 'm' for a Timedelta. Hence, you could test dtype.str[1] == 'M' to detect a Datetime only or dtype.str[1].lower() == 'm' to detect either a Datetime or Timedelta.

>>> dr = pd.date_range('1-1-2017',periods=4,freq='d')
>>> df = pd.DataFrame({ 'i':range(3), 'x':[1.1,2.2,3.3], 
                        'ts':dr[:3], 'td':dr[1:]-dr[:3] })

   i     td         ts    x
0  0 1 days 2017-01-01  1.1
1  1 1 days 2017-01-02  2.2
2  2 1 days 2017-01-03  3.3

>>> for v in df.columns:  
        print( '\ncolumn ' + v + ': ')
        print( 'dtype.str: ', df[v].dtype.str )
        print( 'dtype:     ', df[v].dtype )
        print( 'timestamp? ', df[v].dtype.str[1] == 'M' )

column i: 
dtype.str:  <i8
dtype:      int64
timestamp?  False

column td: 
dtype.str:  <m8[ns]
dtype:      timedelta64[ns]
timestamp?  False

column ts: 
dtype.str:  <M8[ns]
dtype:      datetime64[ns]
timestamp?  True

column x: 
dtype.str:  <f8
dtype:      float64
timestamp?  False
like image 22
JohnE Avatar answered Sep 27 '22 17:09

JohnE