Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

QuickCheck exit status on failures, and cabal integration

I'm trying to understand how to integrate some quickcheck tests with cabal. This gist suggests that the quickCheck function returns non-zero status on failure, but I am not getting that behavior, so using cabal's exitcode-stdio-1.0 test-suite type doesn't seem to work for me unless I want to call error all over my tests.

The cabal user guide also mentions a detailed-1.0 test-suite, but AFAICT this doesn't exist yet. Is that still the case?

It seems from answers like this one that a lot of people are using the test-framework package. That's overkill for me, but is that what I should use?

I'm left kind of unsatisfied by this situation.

Versions of things I'm using:

cabal-install version 0.10.2
using version 1.10.1.0 of the Cabal library
QuickCheck-2.4.1.1
like image 238
jberryman Avatar asked Jan 23 '12 18:01

jberryman


2 Answers

Looking at quickCheck's implementation, it indeed never exits the program. However, you can easily implement this behaviour using quickCheckResult:

import Control.Monad
import Test.QuickCheck
import Test.QuickCheck.Test
import System.Exit

main :: IO ()
main = do
  result <- quickCheckResult prop
  unless (isSuccess result) exitFailure

My understanding is that detailed-1.0 is not considered ready for general use yet, and that exitcode-stdio-1.0 is still the recommended testing solution for now.

like image 185
ehird Avatar answered Nov 17 '22 07:11

ehird


I used test-framework in the latest version of my Decimal package. It was not overkill; it did just what was wanted. Take a look at the complete source code if you want an example of how to use it.

like image 34
Paul Johnson Avatar answered Nov 17 '22 09:11

Paul Johnson