Count items greater than a value in pandas groupby

Question

I have the Yelp dataset and I want to count all reviews which have greater than 3 stars. I get the count of reviews by doing this:

reviews.groupby('business_id')['stars'].count()

Now I want to get the count of reviews which had more than 3 stars, so I tried this by taking inspiration from here:

reviews.groupby('business_id')['stars'].agg({'greater':lambda val: (val > 3).count()})

But this just gives me the count of all stars like before. I am not sure if this is the right way to do it? What am I doing incorrectly here. Does the lambda expression not go through each value of the stars column?

EDIT: Okay I feel stupid. I should have used the sum function instead of count to get the value of elements greater than 3, like this:

reviews.groupby('business_id')['stars'].agg({'greater':lambda val: (val > 3).sum()})

Answer 1

You can try to do :

reviews[reviews['stars'] > 3].groupby('business_id')['stars'].count()

Answer 2

As I also wanted to rename the column and to run multiple functions on the same column, I came up with the following solution:

# Counting both over and under
reviews.groupby('business_id')\
       .agg(over=pandas.NamedAgg(column='stars', aggfunc=lambda x: (x > 3).sum()), 
            under=pandas.NamedAgg(column='stars', aggfunc=lambda x: (x < 3).sum()))\
       .reset_index()

The pandas.NamedAgg allows you to create multiple new columns now that the functionality was removed in newer versions of pandas.