compute time difference of DateTimeIndex

Question

I want to compute the time difference between times in a DateTimeIndex

import pandas as pd
p = pd.DatetimeIndex(['1985-11-14', '1985-11-28', '1985-12-14', '1985-12-28'], dtype='datetime64[ns]')

I can compute the time difference of two times:

p[1] - p[0]

gives

Timedelta('14 days 00:00:00')

But p[1:] - p[:-1] doesn't work and gives

DatetimeIndex(['1985-12-28'], dtype='datetime64[ns]', freq=None)

and a future warning:

FutureWarning: using '-' to provide set differences with datetimelike Indexes is deprecated, use .difference()

Any thought on how how I can (easily) compute the time difference between values in a DateTimeIndex? And why does it work for 1 value, but not for the entire DateTimeIndex?

Answer 1

Convert the DatetimeIndex to a Series using to_series() and then call diff to calculate inter-row differences:

In [5]:
p.to_series().diff()

Out[5]:
1985-11-14       NaT
1985-11-28   14 days
1985-12-14   16 days
1985-12-28   14 days
dtype: timedelta64[ns]

As to why it failed, the - operator here is attempting to perform a set difference or intersection of your different index ranges, you're trying to subtract the values from one range with another which diff does.

when you did p[1] - p[0] the - is performing a scalar subtraction but when you do this on an index it thinks that you're perform a set operation

Answer 2

The - operator is working, it's just not doing what you expect. In the second situation it is acting to give the difference of the two datetime indices, that is the value that is in p[1:] but not in p[:-1]

There may be a better solution, but it would work to perform the operation element wise:

[e - k for e,k in zip(p[1:], p[:-1])]