While writing a test with a value that gets represented as a BigDecimal, I ran into something weird and decided to dig into it. In brief, '0.00009' when rounded to two decimal places is returned as 0.01 instead of 0.00. Really. Here's my script/console capture:
>> bp = BigDecimal('0.09')
=> #<BigDecimal:210fe08,'0.9E-1',4(8)>
>> bp.round(2,BigDecimal::ROUND_HALF_DOWN).to_f
=> 0.09
>> bp = BigDecimal('0.009')
=> #<BigDecimal:210bcf4,'0.9E-2',4(8)>
>> bp.round(2,BigDecimal::ROUND_HALF_DOWN).to_f
=> 0.01
>> bp = BigDecimal('0.0009')
=> #<BigDecimal:2107a8c,'0.9E-3',4(12)>
>> bp.round(2,BigDecimal::ROUND_HALF_DOWN).to_f
=> 0.0
>> bp = BigDecimal('0.00009')
=> #<BigDecimal:2103428,'0.9E-4',4(12)>
>> bp.round(2,BigDecimal::ROUND_HALF_DOWN).to_f
=> 0.01
>> bp = BigDecimal('0.000009')
=> #<BigDecimal:20ff0f8,'0.9E-5',4(12)>
>> bp.round(2,BigDecimal::ROUND_HALF_DOWN).to_f
=> 0.0
Oh, and I get the same results if I use the default mode, like so:
>> bd = BigDecimal('0.00009')
=> #<BigDecimal:2152ed8,'0.9E-4',4(12)>
>> bd.round(2).to_f
=> 0.01
Here are my versions:
ruby 1.8.6 (2008-03-03 patchlevel 114) [i686-darwin9.2.2]
Rails 2.3.4
Has anyone seen anything like this?
No, never seen this before, and it definitely looks like a bug. 0.00009
rounded to two decimal places should definitely be 0.00
.
The ROUND_HALF_DOWN
should not change the behaviour since you're not dealing with midpoint values.
This link has more details.
It seems to be a bug in the 1.8 levels that's been fixed in 1.9. It's a slightly bizarre one in that it only seems to affect numbers with an even number of zeros before the first non-zero digit and only if that digit is 5 or greater.
That appears to be exactly your problem based on the data provided.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With