Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python pandas dataframe slicing by date conditions

I am able to read and slice pandas dataframe using python datetime objects, however I am forced to use only existing dates in index. For example, this works:

>>> data <class 'pandas.core.frame.DataFrame'> DatetimeIndex: 252 entries, 2010-12-31 00:00:00 to 2010-04-01 00:00:00 Data columns: Adj Close    252  non-null values dtypes: float64(1)  >>> st = datetime.datetime(2010, 12, 31, 0, 0) >>> en = datetime.datetime(2010, 12, 28, 0, 0)  >>> data[st:en]             Adj Close Date                  2010-12-31     593.97 2010-12-30     598.86 2010-12-29     601.00 2010-12-28     598.92 

However if I use a start or end date that is not present in the DF, I get python KeyError.

My Question : How do I query the dataframe object for a date range; even when the start and end dates are not present in the DataFrame. Does pandas allow for range based slicing?

I am using pandas version 0.10.1

like image 725
Rishabh Sagar Avatar asked Apr 23 '13 17:04

Rishabh Sagar


People also ask

How do you filter dates in data frames?

strftime() to Filter DataFrame Rows on Dates. You can use df[df['Date']. dt. strftime('%Y-%m')=='2021-11'] method to filter by month.

How do I filter between two dates in Python?

There are two possible solutions: Use a boolean mask, then use df. loc[mask] Set the date column as a DatetimeIndex, then use df[start_date : end_date]


1 Answers

Use searchsorted to find the nearest times first, and then use it to slice.

In [15]: df = pd.DataFrame([1, 2, 3], index=[dt.datetime(2013, 1, 1), dt.datetime(2013, 1, 3), dt.datetime(2013, 1, 5)])  In [16]: df Out[16]:              0 2013-01-01  1 2013-01-03  2 2013-01-05  3  In [22]: start = df.index.searchsorted(dt.datetime(2013, 1, 2))  In [23]: end = df.index.searchsorted(dt.datetime(2013, 1, 4))  In [24]: df.iloc[start:end] Out[24]:              0 2013-01-03  2 
like image 174
waitingkuo Avatar answered Sep 21 '22 13:09

waitingkuo