I have the following query:
-- CTE to remove outliers, e.g. remove the fastest and slowest results
;WITH MinMaxCTE AS
(
SELECT ServerName, CONVERT(VARCHAR(10), UpdatedOn, 101) AS [Date], Version,
MIN(JaguarStartupTime) AS MinStartTime, MAX(JaguarStartupTime) AS MaxStartTime
FROM dbo.MachineConfiguration (NOLOCK)
WHERE DomainLogin NOT IN (SELECT DomainLogin FROM dbo.SupportGroup)
GROUP BY ServerName, CONVERT(VARCHAR(10), UpdatedOn, 101), Version
)
SELECT AVG(mc.JaguarStartupTime) AS AverageTime
, COUNT(*) AS NumEntries
, mc.Version
FROM #Eligible mc (NOLOCK)
JOIN MinMaxCTE cte ON mc.ServerName = cte.ServerName
AND CONVERT(VARCHAR(10), mc.UpdatedOn, 101) = cte.[Date]
AND mc.Version = cte.Version
AND mc.JaguarStartupTime <> cte.MinStartTime
AND mc.JaguarStartupTime <> cte.MaxStartTime
GROUP BY mc.Version
ORDER BY Version DESC, AVG(mc.JaguarStartupTime) ASC
The definition of the #Eligible temp table is
create table #Eligible (
Version nvarchar(50), JaguarStartupTime int,
ServerName nvarchar(50), UpdatedOn datetime )
No matter what condition or aggregation I comment out, I always get the following error: Arithmetic overflow error converting expression to data type int
.
Where can I go from here? How do I debug this further?
EDIT: Sample data
Version JaguarStartupTime ServerName UpdatedOn
6.4.6.082 16040 NewOrleansLA 2012-08-08 12:34:12.330
6.5.1.012 40390 BatonRougeLA 2012-08-08 18:33:17.440
6.5.1.012 48379 HonoluluHI 2012-08-09 04:42:50.453
You need to increase the width of the variable to store this number e.g. making @sample NUMERIC (6,2) will solve this error.
The error "Arithmetic overflow error converting IDENTITY to data type int" comes when the IDENTITY value is inserted into a column of data type int, but the value is out-of-range.
One solution is to change the data type of the IDENTITY column. For example, if it's smallint , change it to int . Or if it's already int , change it to bigint . Another possible solution would be to reset the IDENTITY seed to a lower value.
"Arithmetic overflow: Using operator '*' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '*' to avoid overflow".
Have you tried casting your jaguarstartup times to a bigint in your avg aggregate like so...
AVG(CAST(mc.JaguarStartupTime AS BIGINT))
This should sort out the arithmetic overflow.
To calculate the mean average the server needs to be able to sum all of the ints first, so the datatype you are averaging on needs to able to store the sum of those values, even if the returned answer is within the range of of an int
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With