cabal test in a sandbox

Question

Let's say I have three of my own packages, A B & C, with dependencies on lots of additional packages in Hackage. I'm using cabal 1.18.

• C depends on A & B.
• B depends on A.
• A & B have test suites.

I set up the sandbox like this:

cd /path/to/sandbox
cabal sandbox init
cabal sandbox add-source /path/to/A
cabal sandbox add-source /path/to/B
cabal sandbox add-source /path/to/C

I want to build all the packages, run all test suites on my packages but not dependency packages, showing full test output. What's the best way to do that?

Option 1

cd /path/to/sandbox
cabal install --enable-tests A B C

Problems:

• There's no way to pass --show-details=always to cabal install.
• Test output is hidden in a log file and not shown.
• If the user happened to do cabal install A earlier, A won't get rebuilt and the tests won't get run.

Option 2

cd /path/to/A
cabal sandbox init --sandbox=/path/to/sandbox/.cabal-sandbox
cd /path/to/B
cabal sandbox init --sandbox=/path/to/sandbox/.cabal-sandbox

cd /path/to/A
cabal configure --enable-tests
cabal test --show-details=always
cd /path/to/B
cabal configure --enable-tests
cabal test --show-details=always
cabal install C

Problems:

• This causes A and B libraries to be unnecessarily rebuilt.

Option 3

In the sandbox cabal.config, add the line tests: True.

Problems:

• This will cause tests to run for all dependent packages from Hackage, which is very slow and fails in some cases.

Answer 1

Cabal is really missing functionality here. My plan is to generalize cabal so it has less (or no) concept of a "current package". Right now lots of commands assume that you're in a directory with a .cabal file and you want to do thing to that package. This is less often the case for large, multi-package projects as you've seen.

What I want is for cabal to take a list of targets for most commands, such as build, test, bench, etc. You can the run tests from several packages by

cabal test --show-details=always \
  pkg-dir1:some-test1 pkg-dir1:some-test2 pkg-dir2

(The above example shows that it should be possible to specify just some sections of a package as well.)

I realize that this doesn't help you much now, but at least you know which direction we're moving in.