Menu
declare @nr1 decimal(20,19),
@nr2 decimal(20,19)
set @nr1 = EXP(1.0)
set @nr2 = PI();
print @nr1/@nr2
As EXP and PI are "infinite" numbers you should always have enough decimals to print
The result for this query is 0.865255979432265082
For the query :
declare @nr12 decimal(34,25),
@nr22 decimal(34,25)
set @nr12 = EXP(1.0)
set @nr22 = PI();
print @nr12/@nr22
I get the result : 0.865255
So my question is, why is the first query more precise then the second one? As decimal(p,s) as it is define in msdn tells me that the second query should be more precise.
438
To store numbers that have fixed precision and scale, you use the DECIMAL data type. In this syntax: p is the precision which is the maximum total number of decimal digits that will be stored, both to the left and to the right of the decimal point. The precision has a range from 1 to 38.
In standard SQL, the syntax DECIMAL( M ) is equivalent to DECIMAL( M ,0) . Similarly, the syntax DECIMAL is equivalent to DECIMAL( M ,0) , where the implementation is permitted to decide the value of M . MySQL supports both of these variant forms of DECIMAL syntax. The default value of M is 10.
x), conversion of float values to decimal or numeric is restricted to values of precision 17 digits only.
This link will help: http://msdn.microsoft.com/en-us/library/ms190476.aspx
according to this the scale of the result of a division e1/e2 will be given by this formula max(6, s1 + p2 + 1) and it also includes this note:
probably you will be better using decimal(19,16) given that the scale for exp() and pi() are 16 in both cases.
106
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
