How to keep index in pandas pivot table

Question

Suppose I create a pandas pivot table:

  adults_per_hh= pd.pivot_table(data,index=["hh_id"],values=["adult"],aggfunc=np.sum)
  adults_per_hh.shape
  (1000,1)

I want to keep hh_id as a column in addition to adult. What is the most efficient way to do this?

Answer 1

I think you need reset_index if use pivot_table, because first column is index:

print (data)
   adult  hh_id
0      4      1
1      5      1
2      6      3
3      1      2
4      2      2

print (pd.pivot_table(data,index=["hh_id"],values=["adult"],aggfunc=np.sum))
       adult
hh_id       
1          9
2          3
3          6

adults_per_hh= pd.pivot_table(data,index=["hh_id"],values=["adult"],aggfunc=np.sum)
                .reset_index()
print (adults_per_hh)
   hh_id  adult
0      1      9
1      2      3
2      3      6

Another solution is use groupby and aggregate sum:

adults_per_hh = data.groupby("hh_id")["adult"].sum().reset_index()
print (adults_per_hh)
   hh_id  adult
0      1      9
1      2      3
2      3      6

Timings:

#random dataframe
np.random.seed(100)
N = 10000000
data = pd.DataFrame(np.random.randint(50, size=(N,2)), columns=['hh_id','adult'])
#[10000000 rows x 2 columns]
print (data)

In [60]: %timeit (pd.pivot_table(data,index=["hh_id"],values=["adult"],aggfunc=np.sum).reset_index())
1 loop, best of 3: 384 ms per loop

In [61]: %timeit (data.groupby("hh_id", as_index=False)["adult"].sum())
1 loop, best of 3: 381 ms per loop

In [62]: %timeit (data.groupby("hh_id")["adult"].sum().reset_index())
1 loop, best of 3: 355 ms per loop