Get Stack to pass test suite name as part of --test-arguments

Question

It's possible, with cabal, to set up a continuous build that records test successes/failures in a format many CI systems will accept with a command like:

cabal test '--test-option=--jxml=test-results/$test-suite.xml'

The important part here is that $test-suite is replaced with the name of the test, so that different tests put their results into different files.

When I use stack, all tests get literally the option --jxml=test-results/$test-suite.xml passed to them, so the end result is that the tests overwrite each others' results.

Is there any way to run all my tests with stack so that I can tell each test suite a different place to write their results?

I'd even accept some stack command that parsed the cabal file for me and told me what test suites there are, because then I could script a loop in bash calling each test one at a time.

Answer 1

I'd even accept some stack command that parsed the cabal file for me and told me what test suites there are, because then I could script a loop in bash calling each test one at a time.

If you're willing to stoop to that, stack ide targets will return a list of targets, from which you can do some bashing to get the list of test suites. Something like this:

stack ide targets 2>&1 |
  while IFS=: read pkg type suite; do
    [[ $type = test ]] && stack test ":$suite" --test-arguments="--jxml=test-results/$suite.xml"
  done