Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Decimal (10,9) variable can't hold the number 50 (SQL Server 2008)

This one is pretty straightforward. Why does the code below cause the error below?

declare @dTest decimal(10, 9)
set @dTest = 50

Error:

Msg 8115, Level 16, State 8, Line 3
Arithmetic overflow error converting int to data type numeric.

According to the MSDN documentation on decimal(p, s), p (or 10 in my case) is the "maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point" whereas s (or 9 in my case) is the "maximum number of decimal digits that can be stored to the right of the decimal point."

My number, 50, has only 2 digits total (which less than the maximum 10), and 0 digits to the right of the decimal (which is less than the maximum 9), therefore it should work.

I found this question about essentially the same issue, but no one explained why the documentation seems to conflict with the behavior. It seems like the s dimension is actually being interpreted as the fixed number of digits to the right of the decimal, and being subtracted from the p number, which in my case leaves 10 - 9 = only 1 digit remaining to handle the left side.

Can anyone provide a reasonable way to interpret the documentation as written to match the behavior?

EDIT:

I see some explanations below, but they don't address the fundamental problem with the wording of the docs. I would suggest this change in wording:

For "p (precision)" change "The maximum total number of decimal digits that can be stored" to read "The maximum total number of decimal digits that will be stored".

And for "s (scale)" change "The maximum number of decimal digits that can be stored to the right of the decimal point." to "The number of decimal digits that will be stored to the right of the decimal point. This number is substracted from p to determine the maximum number of digits to the left of the decimal point."

I'm going to submit a bug report to Connect unless some one has a better explanation.

like image 397
Jordan Rieger Avatar asked Aug 30 '12 23:08

Jordan Rieger


2 Answers

10 - 9 is 1. DECIMAL(10, 9) can hold a number in the format 0.000000000. 50 has two digits before the decimal point, and is therefore out of range. You quoted it yourself:

According to the MSDN documentation on decimal(p, s), p (or 10 in my case) is the "maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point" whereas s (or 9 in my case) is the "maximum number of decimal digits that can be stored to the right of the decimal point."

like image 50
Ry- Avatar answered Oct 20 '22 21:10

Ry-


I submitted a bug report to Connect: Misleading documentation on the decimal data type

like image 40
Jordan Rieger Avatar answered Oct 20 '22 22:10

Jordan Rieger