Filtering rows with MultiIndex columns

Question

When creating a DataFrame with MultiIndex columns it seems not possible to select / filter rows using syntax like df[df["AA"]>0.0]. For example:

import pandas as pd
import numpy as np

dates = np.asarray(pd.date_range('1/1/2000', periods=8))
_metaInfo = pd.MultiIndex.from_tuples([('AA', '[m]'), ('BB', '[m]'), ('CC', '[s]'), ('DD', '[s]')], names=['parameter','unit'])

df = pd.DataFrame(randn(8, 4), index=dates, columns=_metaInfo)
print df[df['AA']>0.0]

The result of df["AA"]>0.0 is an indexed DataFrame iso a Timeseries. This probably causes the crash.

When using the same metaInfo as an index for the rows, the situation is different:

df1 = pandas.DataFrame(np.random.randn(4, 6), index=_metaInfo)
print df1[df1["AA"]>0.0]

produces:

[ 1.13268106 -0.06887761  0.68535054  2.49431163 -0.29349413  0.34772553]

which are the elements of row AA larger than zero. This gives only the values of row AA and not of the other columns of the DataFrame.

Is there a workaround? Am I trying to do something I shouldn't?

Answer 1

You can select only the 'AA' column and use it as a filter on the entire df.

Like:

df[df[('AA','[m]')]>0.0]

parameter         AA        BB        CC        DD
unit             [m]       [m]       [s]       [s]
2000-01-01  0.600748 -1.163793 -0.982248 -0.397988
2000-01-03  1.045428  0.365353  0.049152  1.902942
2000-01-06  0.891202  0.021921  1.215515 -1.624741
2000-01-08  0.999217 -1.110213  0.257718 -0.096018