Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas groupby: percentage above threshold

Tags:

python

pandas

I have a DataFrame that I'm looking to use a groupby on but I'm looking for a little bit of an unusual function to aggregate with. I would like to get the percentage of observations in each group above a certain threshold. For example, with a threshold of 0, the DataFrame

df = pd.DataFrame(dict(day=[1, 1, 1, 2, 2, 2, 3, 3, 3, 4], value=[0, 4, 0, 4, 0, 4, 0, 4, 0, 4]))

df
   day  value
0    1      0
1    1      4
2    1      0
3    2      4
4    2      0
5    2      4
6    3      0
7    3      4
8    3      0
9    4      4

should become

df_group = pd.DataFrame(dict(day=[1, 2, 3, 4], value=[.33, .67, .33, 1.0]))

df_group
   day  value
0    1   0.33
1    2   0.67
2    3   0.33
3    4   1.00

I am also working with a fairly large data set, so I'd appreciate taking computation time into account.

like image 848
David Kelley Avatar asked Feb 23 '15 21:02

David Kelley


1 Answers

>>> df.groupby('day')['value'].apply(lambda c: (c>0).sum()/len(c))
day
1      0.333333
2      0.666667
3      0.333333
4      1.000000
Name: value, dtype: float64
like image 159
BrenBarn Avatar answered Oct 28 '22 20:10

BrenBarn