Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I compare two decimals to 10 decimal places?

Tags:

.net

decimal

I'm using decimal type (.net), and I want to see if two numbers are equal. But I only want to be accurate to 10 decimal places. For example take these three numbers. I want them all to be equal.

0.1123456789
0.11234567891
0.11234567899

The 10 decimal limit is coming from my database, so I have to assume that the first number was already rounded, and therefore I can't simply round the others because the last one will round up.

I really just want to truncate the to 10 decimal places, but can't see how to do this either.

like image 905
TheSean Avatar asked Dec 04 '22 14:12

TheSean


1 Answers

The same way you'd compare floating point numbers. Here's some pseudocode, because I don't know the .NET call for absolute value, but it will essentially look like this (modify the constant for the precision needed):

if( Math.Abs( value1 - value2 ) < 0.0000000001 )
{
  // blah blah
}
like image 133
Russell Newquist Avatar answered Dec 31 '22 02:12

Russell Newquist