Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What unit testing frameworks are available for F#

I am looking specifically for frameworks that allow me to take advantage of unique features of the language. I am aware of FsUnit. Would you recommend something else, and why?

like image 653
GregC Avatar asked Apr 14 '11 17:04

GregC


People also ask

Which framework is used for unit testing?

JUnit is an open source framework you can use to write and run tests. It aims to help develop bug-free and reliable code written in Java. JUnit provides test runners that run tests and assertions to test the expected results.

How many types of unit testing are there?

There are 2 types of Unit Testing: Manual, and Automated.

Which of the following is a unit testing framework for C++?

Embunit is designed as a unit testing tool for both developers and testers for software application written in C or C++.


1 Answers

My own unit testing library, Unquote, takes advantage of F# quotations to allow you to write test assertions as plain, statically checked F# boolean expressions and automatically produces nice step-by-step test failure messages. For example, the following failing xUnit test

[<Fact>]
let ``demo Unquote xUnit support`` () =
    test <@ ([3; 2; 1; 0] |> List.map ((+) 1)) = [1 + 3..1 + 0] @>

produces the following failure message

Test 'Module.demo Unquote xUnit support' failed: 

([3; 2; 1; 0] |> List.map ((+) 1)) = [1 + 3..1 + 0]
[4; 3; 2; 1] = [4..1]
[4; 3; 2; 1] = []
false

        C:\File.fs(28,0): at Module.demo Unquote xUnit support()

FsUnit and Unquote have similar missions: to allow you to write tests in an idiomatic way, and to produce informative failure messages. But FsUnit is really just a small wrapper around NUnit Constraints, creating a DSL which hides object construction behind composable function calls. But it comes at a cost: you lose static type checking in your assertions. For example, the following is valid in FsUnit

[<Test>]
let test1 () =
    1 |> should not (equal "2")

But with Unquote, you get all of F#'s static type-checking features so the equivalent assertion would not even compile, preventing us from introducing a bug in our test code

[<Test>] //yes, Unquote supports both xUnit and NUnit automatically
let test2 () =
    test <@ 1 <> "2" @> //simple assertions may be written more concisely, e.g. 1 <>! "2"
    //           ^^^
    //Error 22 This expression was expected to have type int but here has type string

Also, since quotations are able to capture more information at compile time about an assertion expression, failure messages are a lot richer too. For example the failing FsUnit assertion 1 |> should not (equal 1) produces the message

Test 'Test.Swensen.Unquote.VerifyNunitSupport.test1' failed: 
  Expected: not 1
  But was:  1
    C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\VerifyNunitSupport\FsUnit.fs(11,0): at FsUnit.should[a,a](FSharpFunc`2 f, a x, Object y)
    C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\VerifyNunitSupport\VerifyNunitSupport.fs(29,0): at Test.Swensen.Unquote.VerifyNunitSupport.test1()

Whereas the failing Unquote assertion 1 <>! 1 produces the following failure message (notice the cleaner stack trace too)

Test 'Test.Swensen.Unquote.VerifyNunitSupport.test1' failed: 

1 <> 1
false

    C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\VerifyNunitSupport\VerifyNunitSupport.fs(29,0): at Test.Swensen.Unquote.VerifyNunitSupport.test1()

And of course from my first example at the beginning of this answer, you can see just how rich and complex Unquote expressions and failure messages can get.

Another major benefit of using plain F# expressions as test assertions over the FsUnit DSL, is that it fits very well with the F# process of developing unit tests. I think a lot of F# developers start by developing and testing code with the assistance of FSI. Hence, it is very easy to go from ad-hoc FSI tests to formal tests. In fact, in addition to special support for xUnit and NUnit (though any exception-based unit testing framework is supported as well), all Unquote operators work within FSI sessions too.

like image 62
Stephen Swensen Avatar answered Sep 20 '22 13:09

Stephen Swensen