How to create a generator with a fixed list of items for FsCheck

Question

I originally tried to create a generator that have the first 5 elements fixed (and on any test using Prop.forAll the first five would always run), but failed in doing so.

Now I am trying to simplify this by having one generator for random data within a range, and one generator for non-random data, i.e., a fixed sequence. It is similar to Gen.constant, except that instead of one value, it is a sequence of values.

I have this (simplified reproducible example, works with NUnit and xUnit):

[<Property(Verbose = true, MaxTest=5)>]
static member MultiplyIdentityCornerCases () =
    Gen.elements [0L; -1L; 1L; Int64.MinValue; Int64.MaxValue]
    |> Arb.fromGen 
    |> Prop.forAll <| fun x -> x = x * 1L

The output is (no idea where the null comes from):

0:
<null>
9223372036854775807L
1:
<null>
-9223372036854775807L
2:
<null>
-9223372036854775807L
3:
<null>
1L
4:
<null>
-9223372036854775807L
Ok, passed 5 tests.

I'd like the output to contain all the five tests in the sequence, preferably, but not necessarily, in order. I know I can do this with NUnit (or any unit testing system) using a testdata provider, but I wonder whether I can do it with FsCheck (or whether I should, perhaps this is a bad idea).

I think using FsCheck is useful, as for the situation where there's more than one function argument, I want it to exhaustively test all combinations of the corner cases arguments I give it. This is hopefully easier with FsCheck than with a testdata provider.

Answer 1

I'm not aware that that's possible, but you can do this:

open System
open FsCheck
open FsCheck.Xunit

[<Property>]
let MultiplyIdentityCornerCases () =
    Gen.oneof [
        Gen.elements [Int64.MinValue; -1L; 0L; 1L; Int64.MaxValue]
        Arb.generate ]
    |> Arb.fromGen
    |> Prop.forAll <| fun x -> x = x * 1L

Two generators are passed to Gen.oneof, so each of these will generate approximately half of the values.

Gen.elements ought to pick uniformly from all values in the provided sequence, so it'll use e.g. 0L 20% of the time, but only for those half when Gen.oneof uses Gen.elements.

In other words, each of those 'special' values will be generated 50% * 20% = 10% of the time.

By default, a property runs 100 test cases, so on average, it should generate 10 0L values, 10 Int64.MinValue values, and so on. That should often be good enough.

---

If it isn't, you can always do something like this:

open System
open Xunit
open FsCheck
open FsCheck.Xunit
open Swensen.Unquote

[<Theory>]
[<InlineData(Int64.MinValue)>]
[<InlineData(-1L)>]
[<InlineData( 0L)>]
[<InlineData( 1L)>]
[<InlineData(Int64.MaxValue)>]
let MultiplyIdentityCornerCases x = x =! x * 1L

[<Property>]
let MultiplyIdentityCornerCasesProperty x =
    MultiplyIdentityCornerCases x

Here, you define a Parameterized Test using xUnit.net's [<Theory>] feature, and feed it the five corner cases you are concerned with. When you run the tests, the test runner will run those five test cases.

Furthermore, it'll run MultiplyIdentityCornerCasesProperty because it's annotated with [<Property>], and that function simply calls the other function.