Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

using awk and printf not rounding correctly [duplicate]

I have an issue where I'm using printf to round a float to the proper number of decimal points. I'm getting inconsistent results as shown below.

echo 104.45   |  awk '{printf "%.1f\n",$1}'
104.5                               <-- seem to be correct behaviour

echo 104.445  |  awk '{printf "%.2f\n",$1}'
104.44       (should be 104.45)     <-- seems to be INCORRECT behaviour

echo 104.4445 |  awk '{printf "%.3f\n",$1}'
104.445                             <-- seems to be correct behaviour

I've seen examples where float number in calculations may cause problems, but did not expect this with formatting.

like image 856
ced Avatar asked Aug 02 '26 02:08

ced


2 Answers

The number 104.4445 cannot be represented exactly as a binary number. In other words, your computer doesn't know such a number.

# echo 104.4445 | awk '{printf "%.20f\n",$1}'
104.44450000000000500222

# echo 104.445 | awk '{printf "%.20f\n",$1}'
104.44499999999999317879

That's why the former is rounded to 104.445, while the latter is rounded to 104.44 .

The sjsam's answer is relevant only to numbers which can be represented exactly as a binary number, i. e. m/2**n , where m and n are integers and not too big. Changing ROUNDINGMODE to "A" has absolutely no effect on printing 104.45, 104.445, or 104.4445 :

# echo 104.4445  |  awk -v ROUNDMODE="A" '{printf "%.3f\n",$1}'
104.445
# echo 104.4445 | awk '{printf "%.3f\n",$1}'
104.445
# echo 104.445 | awk -v ROUNDMODE="A" '{printf "%.2f\n",$1}'
104.44
# echo 104.445 | awk '{printf "%.2f\n",$1}'
104.44
like image 161
user31264 Avatar answered Aug 03 '26 18:08

user31264


I tried something analogous in Python and got similar results to you:

>>> round(104.445, 2)
104.44
>>> round(104.4445, 3)
104.445

This seems to be run-of-the-mill wonky floating point wonkiness, especially considering that the floating-point representation of 104.445 is less than the actual mathematical value of 104.445:

>>> 104.445 - 104.44
0.0049999999999954525
>>> 104.445 - 104.44 + 104.44
104.445
like image 22
ameed Avatar answered Aug 03 '26 17:08

ameed