Pandas: Selecting rows based on value counts of a particular column

Question

Whats the simplest way of selecting all rows from a panda dataframe, who's sym occurs exactly twice in the entire table? For example, in the table below, I would like to select all rows with sym in ['b','e'], since the value_counts for these symbols equal 2.

df=pd.DataFrame({'sym':['a', 'b', 'b', 'c', 'd','d','d','e','e'],'price':np.random.randn(9)})

                     price sym
    0              -0.0129   a
    1              -1.2940   b
    2               1.8423   b
    3              -0.7160   c
    4              -2.3216   d
    5              -0.0120   d
    6              -0.5914   d
    7               0.6280   e
    8               0.5361   e

df.sym.value_counts()
Out[237]: 
d    3
e    2
b    2
c    1
a    1

Answer 1

I think you can use groupby by column sym and filter values with length == 2:

print df.groupby("sym").filter(lambda x: len(x) == 2)
      price sym
1  0.400157   b
2  0.978738   b
7 -0.151357   e
8 -0.103219   e

Second solution use isin with boolean indexing:

s = df.sym.value_counts()

print s[s == 2].index
Index([u'e', u'b'], dtype='object')

print df[df.sym.isin(s[s == 2].index)]
      price sym
1  0.400157   b
2  0.978738   b
7 -0.151357   e
8 -0.103219   e

And fastest solution with transform and boolean indexing:

print (df[df.groupby("sym")["sym"].transform('size') == 2])
    price sym
1 -1.2940   b
2  1.8423   b
7  0.6280   e
8  0.5361   e

Answer 2

You can use map, which should be faster than using groupby and transform:

df[df['sym'].map(df['sym'].value_counts()) == 2]

e.g.

%%timeit
df[df['sym'].map(df['sym'].value_counts()) == 2]
Out[1]:
1.83 ms ± 23.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

%%timeit
df[df.groupby("sym")["sym"].transform('size') == 2]
Out[2]:
2.08 ms ± 41.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)