Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas slicing multiindex dataframe

I want to slice a multi-index pandas dataframe

here is the code to obtain my test data:

import pandas as pd

testdf = {
    'Name': {
        0: 'H', 1: 'H', 2: 'H', 3: 'H', 4: 'H'}, 'Division': {
            0: 'C', 1: 'C', 2: 'C', 3: 'C', 4: 'C'}, 'EmployeeId': {
                0: 14, 1: 14, 2: 14, 3: 14, 4: 14}, 'Amt1': {
                    0: 124.39, 1: 186.78, 2: 127.94, 3: 258.35000000000002, 4: 284.77999999999997}, 'Amt2': {
                        0: 30.0, 1: 30.0, 2: 30.0, 3: 30.0, 4: 60.0}, 'Employer': {
                            0: 'Z', 1: 'Z', 2: 'Z', 3: 'Z', 4: 'Z'}, 'PersonId': {
                                0: 14, 1: 14, 2: 14, 3: 14, 4: 15}, 'Provider': {
                                    0: 'A', 1: 'A', 2: 'A', 3: 'A', 4: 'B'}, 'Year': {
                                        0: 2012, 1: 2012, 2: 2013, 3: 2013, 4: 2012}}
testdf = pd.DataFrame(testdf)
testdf
grouper_keys = [
    'Employer',
    'Year',
    'Division',
    'Name',
    'EmployeeId',
    'PersonId']

testdf2 = pd.pivot_table(data=testdf,
                              values='Amt1',
                              index=grouper_keys,
                              columns='Provider',
                              fill_value=None,
                              margins=False,
                              dropna=True,
                              aggfunc=('sum', 'count'),
                              )

print(testdf2)

gives:

enter image description here

Now I can get only sum for A or B using

testdf2.loc[:, slice(None, ('sum', 'A'))]

which gives

enter image description here

How can I get both sum and count for only A or B

like image 304
muon Avatar asked Oct 13 '16 15:10

muon


People also ask

How do I convert MultiIndex to single index in pandas?

To revert the index of the dataframe from multi-index to a single index using the Pandas inbuilt function reset_index(). Returns: (Data Frame or None) DataFrame with the new index or None if inplace=True.

Can we do slicing in DataFrame?

Slicing a DataFrame in Pandas includes the following steps:Ensure Python is installed (or install ActivePython) Import a dataset. Create a DataFrame. Slice the DataFrame.


2 Answers

Use xs for cross section

testdf2.xs('A', axis=1, level=1)

enter image description here

Or keep the column level with drop_level=False

testdf2.xs('A', axis=1, level=1, drop_level=False)

enter image description here

like image 174
piRSquared Avatar answered Nov 03 '22 02:11

piRSquared


You can use:

idx = pd.IndexSlice
df = testdf2.loc[:, idx[['sum', 'count'], 'A']]
print (df)
                                                    sum count
Provider                                              A     A
Employer Year Division Name EmployeeId PersonId              
Z        2012 C        H    14         14        311.17   2.0
                                       15           NaN   NaN
         2013 C        H    14         14        386.29   2.0

Another solution:

df = testdf2.loc[:, (slice('sum','count'), ['A'])]
print (df)
                                                    sum count
Provider                                              A     A
Employer Year Division Name EmployeeId PersonId              
Z        2012 C        H    14         14        311.17   2.0
                                       15           NaN   NaN
         2013 C        H    14         14        386.29   2.0
like image 20
jezrael Avatar answered Nov 03 '22 02:11

jezrael