How to display a reason of a failed test property with quickcheck?

Question

What is the best practice to display reasons for a failed property test when it is tested via QuickCheck?

Consider for example:

prop a b = res /= []
   where
      (res, reason) = checkCode a b

Then the a session could look like:

> quickCheck prop
Falsifiable, after 48 tests:
42
23

But for debugging it would be really convenient to show the reason for failure as part of the quickCheck falsifable report.

I have hacked it like this:

prop a b = if res /= [] then traceShow reason False else True
   where
      (res, reason) = checkCode a b

Is there is a better/nicer or more quickcheckish way to do it?

Answer 1

I assume that your "reason" variable contains some kind of test-specific data on what went wrong. You could instead return a "Result", which contains both success/fail/invalid conditions and a string explaining what went wrong. Properties that return Results are handled by QuickCheck in exactly the same way as properties that return Bool.

(edit) Like this:

module QtTest where 

import Test.QuickCheck
import Test.QuickCheck.Property as P


-- Always return success
prop_one :: Integer -> P.Result
prop_one _ = MkResult (Just True) True "always succeeds" False [] []


-- Always return failure
prop_two :: Integer -> P.Result
prop_two n = MkResult (Just False) True ("always fails: n = " ++ show n) False [] []

Note that it is the "Result" type defined in Test.QuickCheck.Property you want.

There are also some combinators defined in Test.QuickCheck.Property which help you compose the Result rather than calling the constructor directly, such as

prop_three :: Integer -> Property
prop_three n = printTestCase ("always fails: n = " ++ show n) False

I guess it would be better style to use those.