how to get rid of awk fatal division by zero error

Question

when ever I am trying to calculate mean and standard deviation using awk i am getting "awk: fatal: division by zero attempted" error.

my command is

awk '{s+=$3} END{print $2"\t"s/(NR)}'  >> mean;
awk '{sum+=$3;sumsq+=$3*$3} END {print $2"\t"sqrt(sumsq/NR - (sum/NR)^2)}' >>sd

does any one know how to solve this ?

Answer 1

Your trouble is that ... you are dividing by zero.

You have two commands:

awk '{s+=$3} END{print $2"\t"s/(NR)}'  >> mean;
awk '{sum+=$3;sumsq+=$3*$3} END {print $2"\t"sqrt(sumsq/NR - (sum/NR)^2)}' >>sd

The first command reads from standard input to EOF. The second command is then run, tries to read standard input, but finds that it is empty, so it has zero records read, so NR is zero, and you are dividing by 0, and crashing.

You will need to deal with both the mean and the standard deviation in a single command.

awk '{s1 += $3; s2 += $3*$3}
     END {  if (NR > 0){
                print $2 "\t" s1 / NR;
                print $2 "\t" sqrt(s2 / NR - (s1/NR)^2);
            }
         }'

This avoids divide-by-zero errors.