bc not executing via script

Question

Hi I have a script which I am using to calculate the standard deviation from a set of values returned from a server.

I can easily retrieve the commands and build a valid string with the command.

bc <<< "scale=10; sqrt((0+((782636-782030)^2)+((781460-782030)^2)+((782492-782030)^2)+((781704-782030)^2)+((781860-782030)^2))/5)"

I create this command by itterating through an array and appending it to the string like so.

CMD='bc <<< "scale=10; sqrt((0'
for i in "${MEMORY[@]}"; do
    CMD=$CMD'+(('$i'-'$MEAN')^2)'
done;
CMD=$CMD')/5)"'

Once I have done this i then try to execute the command string into a variable

SD=`$CMD`   
echo $SD

However I am getting the output

File <<< is unavailable.

Any Ideas?

Answer 1

<<< needs to be parsed by the shell, which happens before parameter expansion. When you try to execute

SD=`$CMD`

the string in $CMD is not reparsed, so <<< is treated as a literal string and passed as an argument to bc. You need something like

BC_EXPRESSION='scale=10; sqrt((0'
for i in "${MEMORY[@]}"; do
    BC_EXPRESSION+="+(($i-$MEAN)^2)"
done
BC_EXPRESSION+=')/5)'

SD=$( bc <<< "$BC_EXPRESSION" )

Answer 2

You should build your bc command in a string and then pass this string to bc.

If I understand correctly, you have an array of numbers memory¹ and a variable mean and you want to compute the standard deviation of memory (where, I guess, mean is the average of memory).

You need to build up strings of the form (X-mean)^2:

sum_terms=0
for i in "${memory[@]}"; do
    sum_terms+="+(($i)-($mean))^2"
done

At this point, the string sum_terms contains the string that expands to something like:

0+((m1)-(mean))^2+((m2)-(mean))^2+ ... +((mn)-(mean))^2

Finally, you want to enclose this in parentheses, prepend sqrt( and append /5) and pass it to bc:

bc <<< "sqrt(($sum_terms)/5)"

All in one:

sum_terms=0
for i in "${memory[@]}"; do
    sum_terms+="+(($i)-($mean))^2"
done
bc -l <<< "sqrt(($sum_terms)/5)"

Notice that I enclosed the $mean term in parentheses, just in case this is a negative number (it would otherwise clash with the preceding negative sign)—and while I was at it, I also enclosed the $i term in parentheses.

---

As a side note, you could also use E((X-E(X))^2)=E(X^2)-E(X)^2 to compute the standard deviation, and also do it in a more generic way:

Given an array memory, compute its standard deviation:

memory=( 782636 781460 782492 781704 781860 )
sum_mem=0 sum_memsq=0
for i in "${memory[@]}"; do
    sum_mem+="+($i)"
    sum_memsq+="+($i)^2"
done
mean=$(bc -l <<< "($sum_mem)/${#memory[@]}")
bc -l <<< "sqrt(($sum_memsq)/${#memory[@]}-($mean)^2)"

---

In fact you don't really need the explicit for loop here, you can leave printf do the job for you:

memory=( 782636 781460 782492 781704 781860 )
printf -v sum_mem '+(%s)' "${memory[@]}"
printf -v sum_memsq '+(%s)^2' "${memory[@]}"
mean=$(bc -l <<< "(0$sum_mem)/${#memory[@]}")
bc -l <<< "sqrt((0$sum_memsq)/${#memory[@]}-($mean)^2)"

---

¹ let me lowercase all your variable names, uppercase variable names are considered bad practice