Select a List Slices of a Pandas Multiindex/Multicolumn DataFrame

Question

Say I have the following multicolumn Pandas DataFrame:

arrays = [['bar', 'bar', 'baz', 'baz', 'foo', 'foo', ],
          ['one', 'two', 'one', 'two', 'one', 'two', ]]

tuples = list(zip(*arrays))
index = pd.MultiIndex.from_tuples(tuples, names=['first', 'second'])
df = pd.DataFrame(np.random.randn(8, 6), columns=arrays)

      bar                 baz                 foo          
      one       two       one       two       one       two
0  1.018709  0.295048 -0.735014  1.478292 -0.410116 -0.744684
1  1.388296  0.019284 -1.298793  1.597739  0.044640 -0.040337
2 -0.151763 -0.424984 -1.322985 -0.350483  0.590343 -2.189122
3 -0.221250 -0.449578 -1.512640  0.077380 -0.485380 -0.687565
4 -0.334315  1.790056  0.245414 -0.236784 -0.788226  0.483709
5 -0.943732  1.437968 -0.114556 -1.098798  0.482486 -1.527283
6 -1.213711  1.573547  0.425109  0.513945  0.731550  1.216149
7  0.709976  1.741406 -0.379932 -1.326460 -1.506532 -0.795053

What is the syntax to select a combination of multiple slices, like selecting ('bar',:) and ('baz':'foo','two')? I know I can do something like:

df.loc[:, [('bar', 'one'), ('baz', 'two')]]

        bar       baz
        one       two
0  1.018709  1.478292
1  1.388296  1.597739
2 -0.151763 -0.350483
3 -0.221250  0.077380
4 -0.334315 -0.236784
5 -0.943732 -1.098798
6 -1.213711  0.513945
7  0.709976 -1.326460

And something like:

print(df.loc[:, ('bar', slice(None))])

        bar          
        one       two
0  1.018709  0.295048
1  1.388296  0.019284
2 -0.151763 -0.424984
3 -0.221250 -0.449578
4 -0.334315  1.790056
5 -0.943732  1.437968
6 -1.213711  1.573547
7  0.709976  1.741406

But something like:

print(df.loc[:, [('bar', slice(None)), ('baz', 'two')]])

Raises a TypeError exception, while

print(df.loc[:, ['bar', ('baz', 'two')]])

raises a ValueError exception.

So what I am after is a simple syntax to create the following with two slices like:

[('bar', slice(None)), ('baz', 'two')]:

        bar                 baz
        one       two       two
0 -1.438018  1.511736  0.186499
1 -0.432313 -0.478824 -0.055930
2  0.995103 -0.181832 -0.257952
3  0.972293  2.580807  1.536281
4 -0.496261  1.038807  0.209853
5  0.788222 -1.325234 -1.328570

Answer 1

I'd like to extend this great answer from @bunji with the pd.IndexSlice[...] method:

In [75]: df.loc[:, pd.IndexSlice[['bar','baz'], 'two']]
Out[75]:
        bar       baz
        two       two
0 -0.037198  0.814649
1  1.272708  1.258576
2  0.405093 -0.243942
3  0.126001  1.751699
4 -0.135793  0.753241
5 -0.433305 -0.192642
6  0.939398  1.356368
7 -0.121508  3.719689

another less performative solution - using chained filter method:

In [78]: df.filter(like='two').filter(regex='(bar|baz)')
Out[78]:
        bar       baz
        two       two
0 -0.037198  0.814649
1  1.272708  1.258576
2  0.405093 -0.243942
3  0.126001  1.751699
4 -0.135793  0.753241
5 -0.433305 -0.192642
6  0.939398  1.356368
7 -0.121508  3.719689

Answer 2

The type error is because you're asking it to look up a list of indices instead of a tuple of indices. Tuples are hashable whereas lists are not so you get an error because it's trying to hash [('bar', slice(None)), ('baz', 'two')]]. Try:

print(df.loc[:, (('bar', slice(None)), ('baz', 'two'))])