Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas: select the first couple of rows in each group

Tags:

python

pandas

I can't solve this simple problem and I'm asking for help here... I have DataFrame as follows and I want to select the first two rows in each group of 'a'

df = pd.DataFrame({'a':pd.Series(['NewYork','NewYork','NewYork','Washington','Washington','Texas','Texas','Texas','Texas']), 'b': np.arange(9)})

df
Out[152]: 
            a  b
0     NewYork  0
1     NewYork  1
2     NewYork  2
3  Washington  3
4  Washington  4
5       Texas  5
6       Texas  6
7       Texas  7
8       Texas  8

that is, I want an output as follows:

            a  b
0     NewYork  0
1     NewYork  1
2  Washington  3
3  Washington  4
4       Texas  5
5       Texas  6

Thanks a lot for the help.

like image 519
user2921752 Avatar asked Dec 19 '13 23:12

user2921752


1 Answers

In pandas 0.13rc, you can do this directly using head (i.e. no need to reset_index):

In [11]: df.groupby('id', as_index=False).head(2)
Out[11]: 
    id   value
0    1   first
1    1  second
3    2   first
4    2  second
5    3   first
6    3   third
9    4  second
10   4   fifth
11   5   first
12   6   first
13   6  second
15   7  fourth
16   7   fifth

[13 rows x 2 columns]

Note: the correct indices, and this is significantly faster than before (with or without reset_index) even with this small example:

# 0.13rc
In [21]: %timeit df.groupby('id', as_index=False).head(2)
1000 loops, best of 3: 279 µs per loop

# 0.12
In [21]: %timeit df.groupby('id', as_index=False).head(2)  # this didn't work correctly
1000 loops, best of 3: 1.76 ms per loop

In [22]: %timeit df.groupby('id').head(2).reset_index(drop=True)
1000 loops, best of 3: 1.82 ms per loop
like image 132
Andy Hayden Avatar answered Nov 04 '22 05:11

Andy Hayden