Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ColdFusion Round function

Today I come across unexpected behavior or lack of knowledge with ColdFusion 9,10,11 Round function here is my scenario

Round(28.5) ---> result is 29 expected
Round(0.285*100) ---> result is 28 not expected
Round(precisionEvaluate(0.285*100)) ---> result is 29 using precisionEvaluate!
Round(Evaluate(0.285*100)) ---> result is 29 using Evaluate!
This is not big decimal, why I would need to use precisionEvaluate or Evaluate on a number?

On farther research I found more interesting behavior
Round(0.285*100) result is 28 --WHY? I'm expecting 29-- !
Round(0.295*100) result is 30 ---- Correct !
Round(0.275*100) result is 28 ---- Correct !
Round(0.185*100) result is 19 ---- Correct !
Round(0.385*100) result is 39 ---- Correct !

What is is big deal with 0.285*100?

like image 700
Click Avatar asked Dec 15 '14 22:12

Click


1 Answers

It's not the precision of the decimal numbers, it's how the underlying floats are stored in Java. This demonstrates:

<cfoutput>
<cfloop array="#[0.275,0.285,0.295]#" index="s">
#s.getClass().getName()#
<cfset f1 = s + 0>
#f1.getClass().getName()#
#f1.toString()#
<cfset f2 = f1*100>
#f2.toString()#
#round(f2)#<br>
</cfloop>
</cfoutput>

Output:

java.lang.String java.lang.Double 0.275 27.500000000000004 28

java.lang.String java.lang.Double 0.285 28.499999999999996 28

java.lang.String java.lang.Double 0.295 29.5 30

I can only assume under the hood CF uses better precision when converting from a string to a float when performing <cfset f1 = s + 0> as there's no dodgy rounding there. However having performed the multiplication step we're getting an accuracy error bleeding in. 28.5 ends up being just shy of 28.5, so rounds to 28 not 29. It's just a binary fraction arithmetic issue.

BTW, there's nothing special about 0.285. A lot of numbers are similarly effected (have a look at the range from 0.005 to 5.05). You just happened to pick a bunch that aren't (other than 0.285).

like image 73
Adam Cameron Avatar answered Sep 19 '22 07:09

Adam Cameron