Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

VBA rounding problem

Tags:

excel

vba

I have this obscure rounding problem in VBA.

a = 61048.4599674847
b = 154553063.208822
c = a + b   
debug.print c
Result: 
154614111.66879 

Here is the question, why did VBA rounded off variable c? I didn't issued any rounding off function. The value I was expecting was 154614111.6687894847. Even if I round off or format variable c to 15 decimal places I still don't get my expected result.

Any explanation would be appreciated.

Edit:

Got the expected results using cDec. I have read this in Jonathan Allen's reply in Why does CLng produce different results?

Here is the result to the test:

a = cDec(61048.4599674847)
b = cDec(154553063.208822)
c = a + b
?c
154614111.6687894847 
like image 374
Grekoz Avatar asked Sep 07 '11 08:09

Grekoz


People also ask

Why is VBA rounding up?

If the number of digits provided is greater than zero then the number is rounded up to the specified decimal place. If the number of digits is provided as input is equal to zero then the number is rounded up to its nearest integer.

How do you round a number in VBA?

Example #1 – VBA Round Function to Round a NumberStep 1: Insert a new module under Visual Basic Editor (VBE). Step 2: Define a new sub-procedure to store a macro in VBE. Step 4: Now, add “& Round (10.9834, 2)” in front of MsgBox command, so that the rounded value will be shown up in the message box along.

Does VBA round up or down?

The Round function in VBA limited in that it will always either round up or round down – depending on the number – so 19.58786 will round UP to 19.59 whereas 19.58246 will round down to 19.58 – the trick is to have a look at the number in 3rd place after the decimal – if that number is 5 or greater, it will always ...

What is CDec in VBA?

CDEC is a built-in data type conversion function available as a VBA function. VBA CDEC function converts a data type from any other data type to a decimal data type. It stands for “Convert to Decimal.”


2 Answers

The reason is the limited precission that can be stored in a floating point variable.
For a complete explanation you shoud read the paper What Every Computer Scientist Should Know About Floating-Point Arithmetic, by David Goldberg, published in the March, 1991 issue of Computing Surveys.

Link to paper

In VBA the default floating point type is Double which is a IEEE 64-bit (8-byte) floating-point number.

There is another type available: Decimal which is a 96-bit (12-byte) signed integers scaled by a variable power of 10
Put simply, this provides floating point numbers to 28 digit precission.

To use in your example:

a = CDec(61048.4599674847)
b = CDec(154553063.208822)
c = a + b   
debug.print c
Result: 
154614111.6687894847 
like image 193
chris neilsen Avatar answered Nov 15 '22 07:11

chris neilsen


Its not obscure, but its not necessarily obvious.

I think you've sort of answered it - but the basic problem is one of the "size" of the values that is how much data can be stored in a variable of a given type.

If (and this is very crude) you count the number of digits in each of the numbers in your first example you will see that you have 15 so whilst the range of values that a float (the default type) can represent is huge the precision is limited to 15 digits (I'm sure someone will be along to correct this, I'll tick the wiki box...)

So when you add the two numbers together it loses the least significant values in order to remain within the allowable precision for a flow.

By doing a cDec you're converting to a different type of variable (decimal) that is capable of greater precision

like image 25
Murph Avatar answered Nov 15 '22 09:11

Murph