Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

haskell - Average floating point error using QuickCheck

I am using QuickCheck-2.5.1.1 to do QA. I am testing two pure functions gold :: a -> Float and f :: a -> Float, where a instances Arbitrary.

Here gold is a reference calculation and f is a variation I am optimizing.

To date, most of my tests using quickcheck have been using tests like \a -> abs (gold a - f a) < 0.0001.

However, I would like to gather statistics along with checking the threshold, since knowing the average error and standard deviation are useful in guiding my design.

Is there any way to use QuickCheck to gather statistics like this?


Concrete example

To give a concrete example of the sort of thing I'm looking for, suppose I have the following two functions for approximating square roots:

-- Heron's method
heron :: Float -> Float
heron x = heron' 5 1
    where
      heron' n est
          | n > 0 = heron' (n-1) $ (est + (x/est)) / 2
          | otherwise = est

-- Fifth order Maclaurin series expansion
maclaurin :: Float -> Float
maclaurin x = 1 + (1/2) * (x - 1) - (1/8)*(x - 1)^2
                + (1/16)*(x - 1)^3 - (5/128)*(x - 1)^4
                + (7/256)*(x - 1)^5

A test for this might be:

test = quickCheck
       $ forAll (choose (1,2))
       $ \x -> abs (heron x - maclaurin x) < 0.02

So what I'd like to know, as a side-effect of the test, is the statistics on abs (heron x - maclaurin x) (such as the mean and standard deviation).

like image 360
Matt W-D Avatar asked Mar 12 '13 14:03

Matt W-D


1 Answers

Thanks to the comments from Chris Kuklewicz and Ingo, I came up with the following that collects the statistics I want in my example:

resultToWeightList :: Result -> [(Double,Int)]
resultToWeightList r = [ (read s, n) | (s,n) <- labels r]

weightListMuSigma :: [(Double,Int)] -> (Double,Double)
weightListMuSigma wlst = (mu,sigma)  
  where 
    (weightSum,weightSqrSum,entryCount) = foldl addEntry (0,0,0) wlst
    addEntry (s,s2,c) (v,w) = (s + (v * w'), s2 + (v**2 * w'), c + w)
      where w' = fromIntegral w
    entryCount' = fromIntegral entryCount
    mu = weightSum / entryCount'
    var = weightSqrSum / entryCount' - mu**2
    sigma = sqrt var

quietCheckResult :: Testable prop => prop -> IO Result
quietCheckResult p = quickCheckWithResult args p
  where args = stdArgs { chatty = False }

test :: IO ()
test = do { r <- quietCheckResult $ forAll (choose (1,2)) test'
          ; let wlst = resultToWeightList r
          ; let (mu,sigma) = weightListMuSigma wlst 
          ; putStrLn $ "Average: " ++ show mu
          ; putStrLn $ "Standard Deviation: " ++ show sigma
          }
   where
     test' x = collect err (err < 0.1)
       where err = abs $ heron x - maclaurin x
like image 171
Matt W-D Avatar answered Oct 20 '22 15:10

Matt W-D