Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to filter a dataframe of dates by a particular month/day?

Tags:

So my code is as follows:

df['Dates'][df['Dates'].index.month == 11]

I was doing a test to see if I could filter the months so it only shows November dates, but this did not work. It gives me the following error: AttributeError: 'Int64Index' object has no attribute 'month'.

If I do

print type(df['Dates'][0])

then I get class 'pandas.tslib.Timestamp', which leads me to believe that the types of objects stored in the dataframe are Timestamp objects. (I'm not sure where the 'Int64Index' is coming from... for the error before)

What I want to do is this: The dataframe column contains dates from the early 2000's to present in the following format: dd/mm/yyyy. I want to filter for dates only between November 15 and March 15, independent of the YEAR. What is the easiest way to do this?

Thanks.

Here is df['Dates'] (with indices):

0    2006-01-01
1    2006-01-02
2    2006-01-03
3    2006-01-04
4    2006-01-05
5    2006-01-06
6    2006-01-07
7    2006-01-08
8    2006-01-09
9    2006-01-10
10   2006-01-11
11   2006-01-12
12   2006-01-13
13   2006-01-14
14   2006-01-15
...
like image 955
jenny Avatar asked Sep 16 '14 16:09

jenny


People also ask

How can I group by month from a date field using Python Pandas?

Output: In the above example, the dataframe is groupby by the Date column. As we have provided freq = 'M' which means month, so the data is grouped month-wise till the last date of every month and provided sum of price column.

How do I select a specific date in Pandas?

In order to select rows between two dates in pandas DataFrame, first, create a boolean mask using mask = (df['InsertedDates'] > start_date) & (df['InsertedDates'] <= end_date) to represent the start and end of the date range. Then you select the DataFrame that lies within the range using the DataFrame.

How do I filter dates by month and year in Excel?

To insert the Auto Filter, select the cell A1 and press the key Ctrl+Shift+L. And filter the data according to the month and year. This is the way we can put the filter by the date field in Microsoft Excel.


1 Answers

Using pd.to_datetime & dt accessor

The accepted answer is not the "pandas" way to approach this problem. To select only rows with month 11, use the dt accessor:

# df['Date'] = pd.to_datetime(df['Date']) -- if column is not datetime yet
df = df[df['Date'].dt.month == 11]

Same works for days or years, where you can substitute dt.month with dt.day or dt.year

Besides that, there are many more, here are a few:

  • dt.quarter
  • dt.week
  • dt.weekday
  • dt.day_name
  • dt.is_month_end
  • dt.is_month_start
  • dt.is_year_end
  • dt.is_year_start

For a complete list see the documentation

like image 195
Erfan Avatar answered Oct 26 '22 15:10

Erfan