I've come across a curious issue in one of my unit tests where I'm getting unexpected rounding results in JavaScript:
(2.005).toFixed(2)
// produces "2.00"
(2.00501).toFixed(2)
// produces "2.01"
Initially I suspected this was a Webkit only issue but it repros in Gecko which implies to me that it is an expected side effect of either ECMA-262 or IEEE-754. I'm assuming the binary representation of 2.005 is ever so slightly less? Or does ECMA-262 specify a round-to-even methodology for toFixed
?
Anyone care to shed some insight as to what is happening under the hood just to give me peace of mind?
Update: thanks for the comments.
I should add, one of the things that made me a little nervous was the comments found in a quick search in Webkit dtoa.cpp
which seemed to imply that there were multiple paths to rounding and the devs weren't really sure how it worked, including a related FIXME
:
https://trac.webkit.org/browser/trunk/Source/WTF/wtf/dtoa.cpp#L1110
Also, not that it means much but IE9 rounds it as I expected, implying that it either isn't part of ECMA-262 or they have a bug.
If the specification hasn't changed since Rev. 6 of the ECMA 262 draft (edition 5.1, March 2011), (2.005).toFixed(2)
must return the string "2.00"
, since a "Number value" is a
primitive value corresponding to a double-precision 64-bit binary format IEEE 754 value
and the interpretation of numeric literals is specified in 7.8.3 and 8.5 to conform to IEEE 754 "round to nearest" mode (with ties rounded to even significand), which for 2.005
results in the value
x = 4514858626438922 * 2^(-51) = 2.00499999999999989341858963598497211933135986328125
In section 15.7.4.5 which deals with toFixed
, the relevant step 8. a. is:
Let
n
be an integer for which the exact mathematical value ofn
÷ 10f –x
is as close to zero as possible. If there are two suchn
, pick the largern
.
and 2.00 - x
is closer to zero than 2.01 - x
, so n
must be 200 here. The conversion to a string proceeds then in the natural way.
Also, not that it means much but IE9 rounds it as I expected, implying that it either isn't part of ECMA-262 or they have a bug.
A bug. Maybe they tried to go the easy way and multiply with 10^digits
and round. x*100
is exactly 200.5
, so that would produce a string of "2.01"
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With