Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to cast computed column with correct decimal/$ result

I have the following computed column in sql server 2008

[Total] AS  CAST ( ((val1/(1000)) * [val2]) AS DECIMAL(18,2))  PERSISTED,

When val1 = 862500 and val2 = 8, the computed value = 6896.00

I need it to be decimal/money where (862500/1000) * 8 = 6900.00 (not 6896.00).

like image 266
genxgeek Avatar asked Dec 27 '22 12:12

genxgeek


1 Answers

BOL says:

Caution: When you use the +, -, *, /, or % arithmetic operators to perform implicit or explicit conversion of int, smallint, tinyint, or bigint constant values to the float, real, decimal or numeric data types, the rules that SQL Server applies when it calculates the data type and precision of the expression results differ depending on whether the query is autoparameterized or not.

Therefore, similar expressions in queries can sometimes produce different results. When a query is not autoparameterized, the constant value is first converted to numeric, whose precision is just large enough to hold the value of the constant, before converting to the specified data type. For example, the constant value 1 is converted to numeric (1, 0), and the constant value 250 is converted to numeric (3, 0).

When a query is autoparameterized, the constant value is always converted to numeric (10, 0) before converting to the final data type. When the / operator is involved, not only can the result type's precision differ among similar queries, but the result value can differ also. For example, the result value of an autoparameterized query that includes the expression SELECT CAST (1.0 / 7 AS float) will differ from the result value of the same query that is not autoparameterized, because the results of the autoparameterized query will be truncated to fit into the numeric (10, 0) data type. For more information about parameterized queries, see Simple Parameterization.

So, you need to convert [val1], 1000 and [val2] to float types:

[Total] AS CAST ( ((CAST ([val1] as float)/CAST (1000 as float)) * CAST ([val2] as float)) AS DECIMAL(18,2)) PERSISTED
like image 89
Alex_L Avatar answered Jan 17 '23 04:01

Alex_L