Can I use Haskell's stack to compile and run _only_ the tests?

Question

When I run a stack test or a stack test <package>:<test-suite> then the output looks something like:

package: configure (lib + exe + test)
package: build (lib + exe + test)
package: copy/register
package: test (suite: tests)

And it ends up basically compiling all my changes twice: once for the exe or lib and a second time for the test.

What I would like is a command like stack test --test-only that would produce something like:

package: configure (test)
package: build (test)
package: copy/register
package: test (suite: tests)

I have already looked through the available command line flags and stack documentation. I have also attempted a google search to see if anyone has talked about something similar.

So my questions are:
1. What is the best currently available way to compile and run only the tests? My best guess on this one is to try putting the tests in a separate cabal package.
2. Is there some reason why stack doesn't or couldn't do this?

Answer 1

I have made some experiments using stack build <pkgname>:test:<testsuite> and found nothing really pleasing when you have all your app/src/main in one directory cabal-project.

Now I did not investigate if this is an issue due to stack using Cabal as a library or whether this is a stack issue.

here are a few issues that might be related

• Turning on test coverage rebuilds all packages #1940
• stack ghci foo:test:bar uses library dependencies unless --test is passed #1845
• Allow stack test to skip certain test suites #1659
• How to specify a subset of tests to run? #2519
• "stack test " does not run tests? #1961

But I guess you'd have to file a bug if noone provides a better answer.

---

A possible but quite ugly solution (in my opinion) is to split test-suite, app and library into separate cabal projects - here is the example folder structure I used for testing.

myproject
├── stackapp            
│   ├── app             
│   │   └── Main.hs     
│   ├── ChangeLog.md    
│   ├── LICENSE         
│   ├── Setup.hs        
│   └── stackapp.cabal  
├── stacksrc            
│   ├── ChangeLog.md    
│   ├── LICENSE         
│   ├── Setup.hs        
│   ├── src             
│   │   └── Lib.hs      
│   └── stacksrc.cabal  
├── stacktest           
│   ├── ChangeLog.md    
│   ├── LICENSE         
│   ├── Setup.hs        
│   ├── src             
│   ├── stacktest.cabal 
│   └── tst             
│       └── Spec.hs     
└── stack.yaml  

stack.yaml

resolver: lts-7.3

packages:
- './stacksrc'
- './stacktest'
- './stackapp'
extra-deps: []
flags: {}
extra-package-dbs: []

Note that you need to include a "dummy" library section in order to make it compile, cabal is picky about cabal files that have neither lib nor exe part.

stacktest.cabal

...
library
  -- dummy
  build-depends:       base >=4.9 && <4.10
  hs-source-dirs:      src
  default-language:    Haskell2010

test-suite tests
  type:          exitcode-stdio-1.0
  main-is:       Spec.hs
  hs-source-dirs: tst
  build-depends: base
               , stacksrc
               , hspec
               , hspec-expectations-pretty-diff

  default-language: Haskell2010

Then you can modify tests and run stack stacktests:test:tests without rebuilding the lib and/or the app part, but stack is intelligent enough designed to rebuild the lib part if you change it before you run the tests.

UPDATE

For future reference here is a link to the opened ticket:

To use stack to compile and run only the tests #2710