Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split a series on time gaps in pandas?

Tags:

python

pandas

Is it possible to split a time series on it's gaps. For example, suppose we had the following:

rng2011 = pd.date_range('1/1/2011', periods=72, freq='H')
rng2012 = pd.date_range('1/1/2012', periods=72, freq='H')
Y = rng2011.union(rng2012)

Is it possible to look for gaps of a year or more, and split the data frame on them?

I imagine this would go something like:

Y.groupby(Y.map(lambda x: x.year))

Except that this splits on the year date, and I'm interested in specifying an interval gap rather than the year attribute of the row.

The application is I've got trip logs from a gps, but no delineation of when one trip ended and another began. I'd like to split on gaps of ten minutes or longer.

like image 299
Maus Avatar asked Dec 20 '12 16:12

Maus


1 Answers

Assuming Y is a column in your dataframe, one way is to use diff and cumsum:

df = DataFrame(Y)
df[1] = df[0].diff() > 600000000000.0 #nanoseconds in ten minutes
df[1] = df[1].cumsum()
df.groupby(1)

Note: If you use the number of nanoseconds in 72 hours it'll split into two groups.

like image 121
Andy Hayden Avatar answered Oct 08 '22 12:10

Andy Hayden