Floating point rounding in shell

Question

$ printf "%0.2f\n" 41.495
41.49
$ printf "%0.2f\n" 41.485
41.49
$ printf "%0.2f\n" 41.475
41.47
$ printf "%0.2f\n" 41.465
41.47
$ printf "%0.2f\n" 41.455
41.46
$ printf "%0.2f\n" 41.445
41.44
$ printf "%0.2f\n" 41.435
41.44
$ printf "%0.2f\n" 41.425
41.42
$ printf "%0.2f\n" 41.415
41.42
$ printf "%0.2f\n" 41.405
41.40

Why are the numbers with an uneven number as the second decimal not correctly rounded and even ones are? Additionally what is wrong with .445 that it never gets rounded?

Answer 1

It has to do with floating-point, but not with double-precision.

When you write

printf "%0.2f\n" 41.495

on your system, printf first rounds 41.495 to the closest x87 80-bit floating-point number[1]. How does that work? First write 41.495 in binary:

b101001.0111 11101011100001010001 11101011100001010001 11101011100001010001 ...

(the separated groups repeat ad infinitum). Now we round this number to have 64 binary digits:

b101001.0111111010111000010100011110101110000101000111101011100001

This is the number that is actually formatted by printf. Written in decimal, it is exactly:

41.4949999999999999990285548534529880271293222904205322265625

as you can see, it is just a little bit less than 41.495, so when printf rounds it to two fractional digits, it rounds down, and 41.49 is printed.

Now look at 41.485; after rounding to 64 binary digits, we get the value:

41.48500000000000000055511151231257827021181583404541015625

which is just a little bit bigger than 41.485, so printf rounds it up.

On my system, there is a warning about this in the printf manage:

Since the floating point numbers are translated from ASCII to floating-point and then back again, floating-point precision may be lost.

---

1. bash doesn't use the x87 format on all operating systems (indeed, it isn't even available on all architectures); on some other systems these values will be interpreted as doubles (and therefore rounded to 53 bits instead of 64), and the results will differ.

Answer 2

I'd lay big odds on it has to do with the IEEE Double precision floating point type. The long-and-short of that is that any decimal number is internally represented with exponent and fraction components, but not in decimals, but in binary. That's not a 100% explanation, and the article explains it much better, but basically floating-point numbers are represented "close" to how they are, not necessarily exactly what you typed in. Thus the rounding can get a bit odd as well.

Read the wiki article. That should help. And if you need exactness, look in to other number representations that don't use this standard.

Answer 3

Your shell or printf command may be using an extended-precision floating point, such as Intel’s 80-bit floating point. printf is implemented directly in some shells and is available as a separate executable, such as in /usr/bin/printf.

The closest single-precision value (in IEEE 754) to 41.495 is 41.494998931884765625. Thus, when the text “41.495” is interpreted as a single-precision value, it stands for exactly 41.494998931884765625. When this value is rounded to two decimal digits after the decimal point, it is 41.49, because the “499…” rounds down.

The closest extended-precision value to 41.495 is 41.4949999999999999990285548534529880271293222904205322265625. Thus, when the text “41.495” is interpreted, it stands for exactly 41.4949999999999999990285548534529880271293222904205322265625. When this is rounded to two decimal digits after the decimal point, it is 41.49.

The closest extended-precision value to 41.485 is 41.48500000000000000055511151231257827021181583404541015625. When rounded, this is 41.49.

The closest extended-precision value to 41.475 is 41.474999999999999998612221219218554324470460414886474609375. When rounded, this is 41.47.

The closest extended-precision value to 41.465 is 41.4650000000000000001387778780781445675529539585113525390625. When rounded, this is 41.47.

The closest extended-precision value to 41.455 is 41.45500000000000000166533453693773481063544750213623046875. When rounded, this is 41.46.

The closest extended-precision value to 41.445 is 41.444999999999999999722444243843710864894092082977294921875. When rounded, this is 41.44.

The closest extended-precision value to 41.435 is 41.4350000000000000012490009027033011079765856266021728515625. When rounded, this is 41.44.

The closest extended-precision value to 41.425 is 41.4249999999999999993061106096092771622352302074432373046875. When rounded, this is 41.42.

The closest extended-precision value to 41.415 is 41.415000000000000000832667268468867405317723751068115234375. When rounded, this is 41.42.

The closest extended-precision value to 41.405 is 41.4049999999999999988897769753748434595763683319091796875. When rounded, this is 41.40.