Pandas TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Int64Index'

Question

I've got some order data that I want to analyse. Currently of interest is: How often has which SKU been bought in which month?

Here a small example:

import datetime
import pandas as pd
import numpy as np

d = {'sku': ['RT-17']}
df_skus = pd.DataFrame(data=d)
print(df_skus)

d = {'date': ['2017/02/17', '2017/03/17', '2017/04/17', '2017/04/18', '2017/05/02'], 'item_sku': ['HT25', 'RT-17', 'HH30', 'RT-17', 'RT-19']}
df_orders = pd.DataFrame(data=d)
print(df_orders)

for i in df_orders.index:
    print("\n toll")
    df_orders.loc[i,'date']=pd.to_datetime(df_orders.loc[i, 'date'])

df_orders = df_orders[df_orders["item_sku"].isin(df_skus["sku"])]
monthly_sales = df_orders.groupby(["item_sku", pd.Grouper(key="date",freq="M")]).size()
monthly_sales = monthly_sales.unstack(0) 

print(monthly_sales)

That works fine, but if I use my real order data (from CSV) I get after some minutes:

TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Int64Index'

That problem comes from the line:

monthly_sales = df_orders.groupby(["item_sku", pd.Grouper(key="date",freq="M")]).size()

Is it possible to skip over the error? I tried a try except block:

try:
    monthly_sales = df_orders.groupby(["item_sku", pd.Grouper(key="date",freq="M")]).size()
    monthly_sales = monthly_sales.unstack(0) 
except:
    print "\n Here seems to be one issue"

Then I get for the print(monthly_sales)

Empty DataFrame
Columns: [txn_id, date, item_sku, quantity]
Index: []

So something in my data empties or brakes the grouping it seems like? How can I 'clean' my data?
Or I'd be even fine with loosing the data of a sale here and there if I can just 'skip' over the error, is this possible?

Answer 1

When reading your CSV, use the parse_dates argument -

df_order = pd.read_csv('file.csv', parse_dates=['date'])

Which automatically converts date to datetime. If that doesn't work, then you'll need to load it in as a string, and then use the errors='coerce' argument with pd.to_datetime -

df_order['date'] = pd.to_datetime(df_order['date'], errors='coerce')

Note that you can pass series objects (amongst other things) to pd.to_datetime`.

Next, filter and group as you've been doing, and it should work.

df_orders[df_orders["item_sku"].isin(df_skus["sku"])]\
     .groupby(['item_sku', pd.Grouper(key='date', freq='M')]).size()

item_sku  date      
RT-17     2017-03-31    1
          2017-04-30    1