Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why are my BigDecimal objects initialized with unexpected rounding errors?

In Ruby 2.2.0, why does:

BigDecimal.new(34.13985572755337, 9)

equal 34.0 but

BigDecimal.new(34.13985572755338, 9)

equal 34.1398557?

Note that I am running this on a 64 bit machine.

like image 413
Andrew Kane Avatar asked Feb 03 '15 09:02

Andrew Kane


1 Answers

Initialize with Strings Instead of Floats

In general, you can't get reliable behavior with Floats. You're making the mistake of initializing your BigDecimals with Float values instead of String values, which introduces some imprecision right at the beginning. For example, on my 64-bit system:

float1 = 34.13985572755337
float2 = 34.13985572755338

# You can use string literals here, too, if your Float can't be properly
# represented. For example:
#
#    BigDecimal.new("34.13985572755337", 9)
#
# would be safer, but Float#to_s works fine with the provided corpus.
bd1 = BigDecimal.new(float1.to_s, 9)
bd2 = BigDecimal.new(float2.to_s, 9)

bd1.to_s
#=> "0.3413985572755337E2"
bd2.to_s
#=> "0.3413985572755338E2"

bd1.to_f == float1
#=> true
bd2.to_f == float2
#=> true

This is one of those cases where the internal representation of the arguments matter. Therefore, your mileage will vary depending on how you initialize your object.

like image 90
Todd A. Jacobs Avatar answered Nov 03 '22 22:11

Todd A. Jacobs