How to calculate number of missing values summed over time dimension in a netcdf file in bash

Question

I have a netcdf file with data as a function of lon,lat and time. I would like to calculate the total number of missing entries in each grid cell summed over the time dimension, preferably with CDO or NCO so I do not need to invoke R, python etc.

I know how to get the total number of missing values

ncap2 -s "nmiss=var.number_miss()" in.nc out.nc

as I answered to this related question: count number of missing values in netcdf file - R

and CDO can tell me the total summed over space with

cdo info in.nc

but I can't work out how to sum over time. Is there a way for example of specifying the dimension to sum over with number_miss in ncap2?

Answer 1

We added the missing() function to ncap2 to solve this problem elegantly as of NCO 4.6.7 (May, 2017). To count missing values through time:

ncap2 -s 'mss_val=three_dmn_var_dbl.missing().ttl($time)' in.nc out.nc

Here ncap2 chains two methods together, missing(), followed by a total over the time dimension. The 2D variable mss_val is in out.nc. The response below does the same but averages over space and reports through time (because I misinterpreted the OP).

Old/obsolete answer:

There are two ways to do this with NCO/ncap2, though neither is as elegant as I would like. Either call assemble the answer one record at a time by calling num_miss() with one record at a time, or (my preference) use the boolean comparison function followed by the total operator along the axes of choice:

zender@aerosol:~$ ncap2 -O -s 'tmp=three_dmn_var_dbl;mss_val=tmp.get_miss();tmp.delete_miss();tmp_bool=(tmp==mss_val);tmp_bool_ttl=tmp_bool.ttl($lon,$lat);print(tmp_bool_ttl);' ~/nco/data/in.nc ~/foo.nc
tmp_bool_ttl[0]=0 
tmp_bool_ttl[1]=0 
tmp_bool_ttl[2]=0 
tmp_bool_ttl[3]=8 
tmp_bool_ttl[4]=0 
tmp_bool_ttl[5]=0 
tmp_bool_ttl[6]=0 
tmp_bool_ttl[7]=1 
tmp_bool_ttl[8]=0 
tmp_bool_ttl[9]=2

or

zender@aerosol:~$ ncap2 -O -s 'for(rec=0;rec<time.size();rec++){nmiss=three_dmn_var_int(rec,:,:).number_miss();print(nmiss);}' ~/nco/data/in.nc ~/foo.nc
nmiss = 0 

nmiss = 0 

nmiss = 8 

nmiss = 0 

nmiss = 0 

nmiss = 1 

nmiss = 0 

nmiss = 2 

nmiss = 1 

nmiss = 2