Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Resample intraday pandas DataFrame without add new days

I want to downsample some intraday data without adding in new days

df.resample('30Min')

Will add weekends etc which is undesirable. Is there anyway around this?

like image 408
Dave Anderson Avatar asked Feb 15 '13 16:02

Dave Anderson


1 Answers

A combined groupby/resample might work:

In [22]: dates = pd.date_range('01-Jan-2014','11-Jan-2014', freq='T')[0:-1]
    ...: dates = dates[dates.dayofweek < 5]
    ...: s = pd.TimeSeries(np.random.randn(dates.size), dates)
    ...: 

In [23]: s.size
Out[23]: 11520

In [24]: s.groupby(lambda d: d.date()).resample('30min').size
Out[24]: 384

In [25]: s.groupby(lambda d: d.date()).resample('30min')
Out[25]: 
2014-01-01  2014-01-01 00:00:00    0.202943
            2014-01-01 00:30:00   -0.466010
            2014-01-01 01:00:00    0.029175
            2014-01-01 01:30:00   -0.064492
            2014-01-01 02:00:00   -0.113348
            2014-01-01 02:30:00    0.100408
            2014-01-01 03:00:00   -0.036561
            2014-01-01 03:30:00   -0.029578
            2014-01-01 04:00:00   -0.047602
            2014-01-01 04:30:00   -0.073846
            2014-01-01 05:00:00   -0.410143
            2014-01-01 05:30:00    0.143853
            2014-01-01 06:00:00   -0.077783
            2014-01-01 06:30:00   -0.122345
            2014-01-01 07:00:00    0.153003
...
2014-01-10  2014-01-10 16:30:00   -0.107377
            2014-01-10 17:00:00   -0.157420
            2014-01-10 17:30:00    0.201802
            2014-01-10 18:00:00   -0.189018
            2014-01-10 18:30:00   -0.310503
            2014-01-10 19:00:00   -0.086091
            2014-01-10 19:30:00   -0.090800
            2014-01-10 20:00:00   -0.263758
            2014-01-10 20:30:00   -0.036789
            2014-01-10 21:00:00    0.041957
            2014-01-10 21:30:00   -0.192332
            2014-01-10 22:00:00   -0.263690
            2014-01-10 22:30:00   -0.395939
            2014-01-10 23:00:00   -0.171149
            2014-01-10 23:30:00    0.263057
Length: 384

In [26]: np.unique(_25.index.get_level_values(1).minute)
Out[26]: array([ 0, 30])

In [27]: np.unique(_25.index.get_level_values(1).dayofweek)
Out[27]: array([0, 1, 2, 3, 4]) 
like image 104
Dave Hirschfeld Avatar answered Oct 13 '22 22:10

Dave Hirschfeld