Testcases for NUnit 3 tests in F#

Question

I am trying to set up a test suite for an F# project using NUnit. It seems that especially when testing things like parsers and type checkers one typically has a list of valid input data and a list of invalid data. The tests itself are practically identical, so I am looking for a clever way to avoid writing a test function for each and every data item and instead seperate the test function from the data. Apparently, there seems to be something called test cases for that but I am having a hard time to find comprehensive documentation for the usage of NUnit 3 with F# in general and specifically a best practice example for my scenario.

Any pointers and hints are greately appreaciated!

Answer 1

This is an updated answer for NUnit 3.x since my original answer showed a NUnit 2.x example.

The examples below are not meant to be comprehensive, but enough to get you over the threshold and up an running. The things of note are that the test functions are written using argument list instead of currying. Also there are several ways to generate test data using NUnit 3.x attributes, e.g. Pairwise, but sadly none of the attributes available know how to generate test data for discriminated unions.

Also FSUnit is not needed and I didn't try to make it work as the difference between NUnint 2.x and 3.x are so dramatic that I was happy just to get the following examples working. Maybe in the future I will update this answer.

namespace NUnit3Demo

open NUnit.Framework

module MyTest = 
    // ----------------------------------------------------------------------

    [<Pairwise>]
    let pairWiseTest([<Values("a", "b", "c")>] (a : string), [<Values("+", "-")>] (b :string), [<Values("x", "y")>] (c : string))
        = printfn "%A %A %A" a b c

    // ----------------------------------------------------------------------

    let divideCases1 =
        [
            12, 3, 4
            12, 2, 6
            12, 4, 3
        ] |> List.map (fun (q, n, d) -> TestCaseData(q,n,d))

    [<TestCaseSource("divideCases1")>]
    let caseSourceTest1(q:int, n:int, d:int) =
        Assert.AreEqual( q, n / d )

    // ----------------------------------------------------------------------

    let divideCases2 =
        seq {
            yield (12, 3, 4)
            yield (12, 2, 6)
            yield (12, 4, 3)
        }

    [<TestCaseSource("divideCases2")>]
    let caseSourceTest2(q:int, n:int, d:int) =
        Assert.AreEqual( q, n / d )

    // ----------------------------------------------------------------------

    [<TestCase(12,3,4)>]
    [<TestCase(12,2,6)>]
    [<TestCase(12,4,3)>]
    let testCaseTest(q:int, n:int, d:int) =
        Assert.AreEqual( q, n / d )

    // ----------------------------------------------------------------------

    let evenNumbers : int [] = [| 2; 4; 6; 8 |]

    [<TestCaseSource("evenNumbers")>]
    let caseSourceTest3 (num : int) =
        Assert.IsTrue(num % 2 = 0)

Leaving Original Answer since it was noted in other answer by OP.

The following example was written 3 years ago using NUnit 2.x so it is a bit dated but should give you an ideal.

You create an array of the test data, then index into the array to pull out the test values and expected results. The nice thing about this is that you don't wind up writing lots of individual test for a function.

This comes from a project some of us did years ago.

open NUnit.Framework
open FsUnit

let private filterValues : (int list * int list)[] = [| 
    (
        // idx 0
        // lib.filter.001
        [],
        []
    ); 
    (
        // idx 1
        // lib.filter.002
        [-2],
        [-2]
    );
    (
        // idx 2
        // lib.filter.003
        [-1],
        []
    );
    (
        // idx 3
        // lib.filter.004
        [0],
        [0]
    );
    (
        // idx 4
        // lib.filter.005
        [1],
        []
    );
    (
        // idx 5
        // lib.filter.006
        [1; 2],
        [2]
    );
    (
        // idx 6
        // lib.filter.007
        [1; 3],
        []
    );
    (
        // idx 7
        // lib.filter.008
        [2; 3],
        [2]
    );
    (
        // idx 8
        // lib.filter.009
        [1; 2; 3],
        [2]
    );
    (
        // idx 9
        // lib.filter.010
        [2; 3; 4],
        [2; 4]
    );
    |]

[<Test>]
[<TestCase(0, TestName = "lib.filter.01")>]
[<TestCase(1, TestName = "lib.filter.02")>]
[<TestCase(2, TestName = "lib.filter.03")>]
[<TestCase(3, TestName = "lib.filter.04")>]
[<TestCase(4, TestName = "lib.filter.05")>]
[<TestCase(5, TestName = "lib.filter.06")>]
[<TestCase(6, TestName = "lib.filter.07")>]
[<TestCase(7, TestName = "lib.filter.08")>]
[<TestCase(8, TestName = "lib.filter.09")>]
[<TestCase(9, TestName = "lib.filter.10")>]
let ``List filter`` idx = 
    let (list, _) = filterValues.[idx]
    let (_, result) = filterValues.[idx]
    List.filter (fun x -> x % 2 = 0) list 
    |> should equal result
    filter (fun x -> x % 2 = 0) list 
    |> should equal result

IIRC the problem with using NUnit with F# is to remember to use <> in the right location.

Answer 2

In NUnit3 there is TestCaseSource and TestCaseData and for the best practices part I added FsUnit:

namespace NUnit3Demo

open NUnit.Framework
open FsUnit

[<TestFixture>]
module MyTest = 

    let methodToBeTested s = 
        if String.length s > 3 then failwith "Something's wrong"
        else String.length s

    let validData =
        [
            TestCaseData("   ").Returns(3)
            TestCaseData("").Returns(0)
            TestCaseData("a").Returns(1)
        ]

    let invalidData =
        [
            "    "
            "abcd"
            "whatever"
        ]

    let otherInvalidData =
        [
            "just"
            "because"
        ]

    [<TestCaseSource("invalidData");
      TestCaseSource("otherInvalidData")>]
    let ``More than 3 characters throws`` s = 
        (fun () -> methodToBeTested s |> ignore)
        |> should throw typeof<System.Exception>

    [<TestCaseSource("validData")>]
    let ``Less than 4 characters returns length`` s = 
        methodToBeTested s

Note that TestCaseData can take and return arbitrary objects (obviously they should match the test signatures). Also, the data can be written even nicer:

let validData =
    [
        "   ", 3
        "",    0
        "a",   1
    ] |> List.map (fun (d, r) -> TestCaseData(d).Returns r)