Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is it in vb.net if I assign a number to a single variable it does not equal the same value

I have been searching the internet trying to understand in its simplist form why this behavior happens.

Dim mysingle As Single = 456.11
Dim mybool As Boolean = mysingle = 456.11

In the lines above mybool becomes false. I found this behavior when putting the single into a double I found extra digits showing. The .net documentations states a single is an approimate value :S

I gatehr a single is a 32bit floating point number? But why are extra digits appearing when I have explicitly said what the number is.. surely the memory should store that numbers either side of my number are 0 to fill up the memory location?!

My brain is fried on this one :(

like image 690
Adam Avatar asked Nov 02 '10 16:11

Adam


People also ask

What does <> mean in VB net?

<> in VB.NET means "not equal to". It can be used with the normal oprands as well as in comparision with the items when compared with the datas fetched with the data reader (from database). Follow this answer to receive notifications.

What is a float in VB net?

Floating-point ( Single and Double ) numbers have larger ranges than Decimal numbers but can be subject to rounding errors. Floating-point types support fewer significant digits than Decimal but can represent values of greater magnitude.


2 Answers

The value being compared in the second statement isn't considered a Single, rather it's being treated as a Double. Since you're using VB.NET you can suffix it with a ! to force it to a Single and this will return True:

Dim mysingle As Single = 456.11
Dim mybool As Boolean = mysingle = 456.11!
like image 186
Ahmad Mageed Avatar answered Nov 15 '22 06:11

Ahmad Mageed


I recommend reading: What Every Computer Scientist Should Know About Floating-Point Arithmetic.

It explains the issues with precision in floating point math in detail. For a simpler variation on the above, see The Floating Point Guide.

like image 26
Reed Copsey Avatar answered Nov 15 '22 07:11

Reed Copsey