HUnit/QuickCheck with Continuous Integration

Question

Are there any extensions to HUnit or QuickCheck that allow a continuous integration system like Bamboo to do detailed reporting of test results?

So far, my best idea is to simply trigger the tests as part of a build script, and rely on the tests to fail with a non-zero exit code. This is effective for getting attention when a test fails, but confuses build failures with test failures and requires wading through console output to determine the problem's source.

If this is the best option with current tools, my thought is to write a reporting module for HUnit that would produce output in the JUnit XML format, then point the CI tool at it as though it were reporting on a Java project. This seems somewhat hackish, though, so I'd appreciate your thoughts both on existing options and directions for new development.

Answer 1

The test-framework package provides tools for integrating tests using different testing paradigms, including HUnit and QuickCheck, and its console test runner can be passed a flag that makes it produce JUnit-compatible XML. We use it with Jenkins for continuous integration.

Invocation example:

$ ./test --jxml=test-results.xml

Answer 2

I've just released a package which generates test-suites based off modules containing quickCheck properties: http://hackage.haskell.org/package/tasty-integrate

This is one step above test-framework/tasty at the moment, as it forcefully pulls/aggregates them off the filesystem, instead of relying upon per-file record keeping. I hope this helps your CI process.