How to Run Multiple Test Files with Haskell Stack Project

Question

I want to setup an existing Haskell project with Stack. The existing project uses multiple files under a test directory; these separate test files by default, Stack (or cabal?) appears to utilize a single test/Spec.hs for testing. How can I continue to use multiple files with this project?

NOTE: I'm learning Haskell, and this project approaches my learning from a "kata" approach. So tests are isolate to focus on one aspect of the language at a time.

Answer 1

Here is a setup for a directory structure like this

> tree                                                                                        . ├── example.cabal ├── app │   └── Main.hs ├── ChangeLog.md ├── LICENSE ├── Setup.hs ├── src │   ├── A │   │   └── C.hs │   ├── A.hs │   └── B.hs ├── stack.yaml └── tst     ├── integration     │   └── Spec.hs     └── unit         ├── A         │   └── CSpec.hs         ├── ASpec.hs         ├── BSpec.hs         └── Spec.hs 

you want to have integration tests that are separate from the usual unit tests and several sub-modules that correspond to each module in your src-folder

first of all you need to add the test suites to your

example.cabal file

name:                example ... -- copyright: -- category: build-type:          Simple extra-source-files:  ChangeLog.md cabal-version:       >=1.10  executable testmain   main-is:       Main.hs   hs-source-dirs: app   build-depends: base                , example  library   exposed-modules:     A.C,A,B   -- other-modules:   -- other-extensions:   build-depends:       base >=4.9 && <4.10   hs-source-dirs:      src   default-language:    Haskell2010  test-suite unit-tests   type:          exitcode-stdio-1.0   main-is:       Spec.hs   hs-source-dirs: tst/unit   build-depends: base                , example                , hspec                , hspec-discover                , ...  test-suite integration-tests   type:          exitcode-stdio-1.0   main-is:       Spec.hs   hs-source-dirs: tst/integration   build-depends: base                , example                , hspec                , ... 

put the following in your tst/unit/Spec.hs it is from hspec-discover and it discovers (hence the name) all modules of the form ...Spec.hs and executes the spec function from each of those modules.

tst/unit/Spec.hs

{-# OPTIONS_GHC -F -pgmF hspec-discover #-} 

just this single line

Other test files

then add your unit tests in your ASpec.hs, and others in BSpec.hs,CSpec.hs and your Spec.hs in the tst/integration folder

module ASpec where  import Test.Hspec import A  spec :: Spec spec = do   describe "Prelude.head" $ do     it "returns the first element of a list" $ do       head [23 ..] `shouldBe` (23 :: Int)      it "returns the first element of an *arbitrary* list" $       property $ \x xs -> head (x:xs) == (x :: Int)      it "throws an exception if used with an empty list" $ do       evaluate (head []) `shouldThrow` anyException 

you can then compile and run your tests with

$> stack test # now all your tests are executed $> stack test :unit-tests # now only the unit tests run $> stack test :integration-tests # now only the integration tests run 

Sources

You can find all the examples at https://hspec.github.io, if you want to know more about hspec-style testing I guess it would be best to start there. For the stack - go to https://haskellstack.org - there is some information about testing/benchmarking there - I mean about running tests and benchmarks.

For different testing style in haskell see HUnit, QuickCheck, Smallcheck, doctests (If I forgot one, my dearest apologies - those are the ones that I use regularly as well).

Answer 2

Here is a solution with just stack and HUnit. Nothing against hspec, htf, tasty, etc etc, but likewise there is not much glue needed even without those, if you're already using HUnit. It doesn't require editing the cabal file. The original question implies use of hspec, so the @epsilonhalbe is still closer on that criteria.

. ├─stack.yaml ├─package.yaml ├─src/ |   ├─A.hs |   ├─B.hs |   ├─Main.hs ├─tst/ |   ├─ATest.hs |   ├─BTest.hs |   ├─Main.hs 

Example package.yaml file:

name:                example version:             0.1.0.0  dependencies: - HUnit >= 1.6.1.0 && < 2  library:   source-dirs: src  executables:   example-exe:     main:                Main.hs     source-dirs:         src     dependencies:     - example  tests:   example-test:     main:                Main.hs     source-dirs:         tst     dependencies:     - example     - HUnit 

In ATest.hs and BTest.hs declare list of tests called huTests in the usual HUnit way, eg,

huTests = ["egTest" ~: "a" ~=? "a"].

Then tst/Main.hs has the glue in a common HUnit idiom (see eg this answer):

import ATest (huTests) import BTest (huTests)  import System.Exit import Test.HUnit  main :: IO () main = do     results <- runTestTT $                     test (ATest.huTests ++ BTest.huTests)     if errors results + failures results == 0 then         putStrLn "Tests passed."     else         die "Tests failed."