Pandas - convert dataframe multi-index to datetime object

Question

Consider an input file, b.dat:

string,date,number
a string,2/5/11 9:16am,1.0
a string,3/5/11 10:44pm,2.0
a string,4/22/11 12:07pm,3.0
a string,4/22/11 12:10pm,4.0
a string,4/29/11 11:59am,1.0
a string,5/2/11 1:41pm,2.0
a string,5/2/11 2:02pm,3.0
a string,5/2/11 2:56pm,4.0
a string,5/2/11 3:00pm,5.0
a string,5/2/14 3:02pm,6.0
a string,5/2/14 3:18pm,7.0

I can group monthly totals like so:

b=pd.read_csv('b.dat')
b['date']=pd.to_datetime(b['date'],format='%m/%d/%y %I:%M%p')
b.index=b['date']
bg=pd.groupby(b,by=[b.index.year,b.index.month])
bgs=bg.sum()

The index of the grouped totals looks like:

bgs

            number
2011 2       1
     3       2
     4       8
     5      14
2014 5      13

bgs.index

MultiIndex(levels=[[2011, 2014], [2, 3, 4, 5]],
       labels=[[0, 0, 0, 0, 1], [0, 1, 2, 3, 3]])

I'd like to reformat the index into date time format (days can be first of month).

I've tried the following:

bgs.index = pd.to_datetime(bgs.index)

and

bgs.index = pd.DatetimeIndex(bgs.index)

Both fail. Does anyone know how I can do this?

Answer 1

Consider resample by 'M' rather than grouping by attributes of the DatetimeIndex:

In [11]: b.resample('M', how='sum').dropna()
Out[11]:
            number
date
2011-02-28       1
2011-03-31       2
2011-04-30       8
2011-05-31      14
2014-05-31      13

Note: you have to drop the NaN if you don't want the months in between.

Answer 2

You can create a column from the index via the date calculation you want, then set that as the index:

bgs['expanded_date'] = bgs.index.map(lambda x: datetime.date(x.year, x.month, 1))
bgs.set_index('expanded_date')