I am using VB.NET 2010. I ran into an issue that at first I thought was a database issue since the value was being pulled from a DB2 table. However, it appears to be in .NET itself. I determined this by running the following two lines of code:
Dim sAmount As Single
sAmount = CType("212639.04", Single)
When I then look at sAmount, the value is 212639.047.
My question: Where is the 7 coming from?
The trouble is you're using Single which is a floating-pont data type, which doesn't work well with fractions that aren't binary fractions.
Floating-Point Expressions Do Not Compare as Equal
When you work with floating-point numbers (Single Data Type (Visual Basic) and Double Data Type (Visual Basic)), remember that they are stored as binary fractions. This means they cannot hold an exact representation of any quantity that is not a binary fraction (of the form k / (2 ^ n) where k and n are integers). For example, 0.5 (= 1/2) and 0.3125 (= 5/16) can be held as precise values, whereas 0.2 (= 1/5) and 0.3 (= 3/10) can be only approximations.
Because of this imprecision, you cannot rely on exact results when you operate on floating-point values. In particular, two values that are theoretically equal might have slightly different representations.
To compare floating-point quantities
Calculate the absolute value of their difference by using the Abs method of the Math class in the System namespace. Determine an acceptable maximum difference, such that you can consider the two quantities to be equal for practical purposes if their difference is no larger. Compare the absolute value of the difference to the acceptable difference.
Alternatively double will return a number with greater precision.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With