How to count number of index or Null values in Pandas dataframe group

Question

Its always the things that seem easy that bug me. I am trying to get a count of the number of non-null values of some variables in a Dataframe grouped by month and year. So I can do this which works fine

counts_by_month=df[variable1, variable2].groupby([lambda x: x.year,lambda x: x.month]).count()

But I REALLY want to know is how many of those values in each group are NaNs. So I want to count the Nans in each variable too so that I can calculate the percentage data missing in each group. I can not find a function to do this. or maybe I could get to the same end by counting the total items in the group. Then the NaNs would be Total - 'Non-Null values'

I have been trying to find out if I can somehow count the index values but I haven't been able to do so. Any assistance on this greatly appreciated. Best wishes Jason

Answer 1

df.isnull().sum()

Faster, and doesn't need a custom function :)