Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

SQL Server sum of float should give value of 1 but filter says otherwise

I have a table with a field's datatype set to float. the values for a given set of records should give a sum of 1 and the field in the select does return 1 however, a HAVING clause states otherwise.

Here are the exact values I have in my table and as you can see, this example does the same thing. Why does this sum up to more than 1 ? I am lost!

with example as (
    SELECT 'Code' as Code, cast(0.462 as float) as perc
    UNION ALL
    SELECT 'Code' as Code, cast(0.116 as float) as perc
    UNION ALL
    SELECT 'Code' as Code, cast(0.181 as float) as perc
    UNION ALL
    SELECT 'Code' as Code, cast(0.053 as float) as perc
    UNION ALL
    SELECT 'Code' as Code, cast(0.188 as float) as perc
)

SELECT 
    Code,
    SUM(perc)
FROM
    example
GROUP BY Code
HAVING SUM(perc) > 1
like image 440
David Laplante Avatar asked Dec 22 '22 00:12

David Laplante


2 Answers

I think you mis-understand floats. I don't mean that in a patronising way, most people mis-understand floats, unfortunately.

Floating point numbers are able to "re-scale" themselves. Being able to hold huge numbers, or tiny numbers, all in the same data-type. That doesn't mean they are 'infinitely precise'. It just means that they have an extreme range that they can cover. They do, however, pay a price for that flexibility. They have a rounding error due to them being binary approximations of decimal numbers.

If you want to know more about it, there is plenty on the web. It messes with people's heads at first, but once you understand you'll be better, safer and wiser for it.


So, for your case, if you want to preserve the exact value, don't use a floating point data type. Use a fixed point data type instead.

If you change from FLOAT to something like DECIMAL(9,4) then you won't get the rounding errors associated with floating point numbers.

You won't be able to store 0.12345 in it though, you'd need to specify DECIMAL(9,5) for that. But you will be always guaranteed that any number you can store in it will be stored exactly, and not approximately.

like image 148
MatBailie Avatar answered Apr 06 '23 03:04

MatBailie


Because FLOAT its not an exact numeric type, so there are floating point approximations. Try using DECIMAL or NUMERIC:

with example as (
SELECT 'Code' as Code, cast(0.462 as NUMERIC(5,3)) as perc
UNION ALL
SELECT 'Code' as Code, cast(0.116 as NUMERIC(5,3)) as perc
UNION ALL
SELECT 'Code' as Code, cast(0.181 as NUMERIC(5,3)) as perc
UNION ALL
SELECT 'Code' as Code, cast(0.053 as NUMERIC(5,3)) as perc
UNION ALL
SELECT 'Code' as Code, cast(0.188 as NUMERIC(5,3)) as perc
)

SELECT 
    Code,
    SUM(perc)
FROM
    example
GROUP BY Code
HAVING SUM(perc) > 1
like image 33
Lamak Avatar answered Apr 06 '23 02:04

Lamak