How do I make NUnit run F# tests not exported by a module

Question

I have written a large module in F# that happens to have a trivial interface. The module contains about 1000 lines of code, 50 unit tests, and exports just one easily understood function.

The natural thing to do next is to write a tiny fsi file. This has numerous advantages including preventing namespace pollution, providing an obvious place for the documentation, making sure that if anyone decides to reuse the internals they will have an incentive to cleanly factor them out, and no doubt many others. I am sure I am preaching to the choir here, but still felt it is worth explaining why I feel it's helpful to have the fsi file.

Now the problem. NUnit won't run the unit tests any more, recalcitrantly claiming they are not public. Well, that would be because they are not in any way a part of the interface. I don't particularly want to add them to the interface despite that, seeing as it would mean updating it every time I added another test, and also that it would bloat the fsi file by an order of magnitude.

I suppose a trivial workaround is to move the code somewhere else, import it into a tiny .fs file, and just forward the one function. With a bit of luck everyone will agree that is simply revolting. Is there a better way please?

Edit: many thanks to everyone who responded. I upvoted both answers. I would have liked to split the bounty, however as that does not appear to be possible I will (somewhat arbitrarily) accept Tomas's answer.

Answer 1

If you're adding an fsi file to specify the visibility of the modules and functions in your source, then you'll need to include declarations of all functions that should be publicly accessible. This means that if NUnit requires tests to be public functions, you'll need to include them in fsi file.

However, there is also another way to specify visibility in F# - instead of using fsi file, you can just add appropriate visibility modifiers to your declarations. This way, you can hide all the implementation details and export only the main function and tests:

namespace MyLibrary
open NUnit.Framework

// Implementation details can be in this module
// (which will not be visible outside of the library)
module private Internal = 
  let foo n = n * 2
  let bar n = n + 1

// A public module can contain the public API (and use internal implementation)    
module public MyModule = 
  open Internal
  let doWork n = foo (bar n)

// To make the tests visible to NUnit, these can be placed in a public module
// (but they can still access all functions from 'Internal')
module public Tests = 
  open MyModule

  [<Test>]
  let ``does work for n = 1``() = 
    Assert.Equals(doWork 1, 4) 

Compared with using fsi files, this has the disadvantage that you don't have a separate file that nicely describes only the important parts of your API. However, you'll get what you need - hide the implementation details and expose only a single function and the tests.

Answer 2

Approach

You could resort to using reflection to invoke your private test methods: you'd have a single public NUnit test method which loops over all private methods in the the assembly invoking those with the Test attribute. The big down-side to this approach is that you can only see one failing test method at a time (but maybe you could look into something creative like using parameterized tests to fix this).

Example

Program.fsi

namespace MyNs

module Program =
    val visibleMethod: int -> int

Program.fs

namespace MyNs

open NUnit.Framework

module Program =
    let implMethod1 x y =
        x + y

    [<Test>]
    let testImpleMethod1 () =
        Assert.AreEqual(implMethod1 1 1, 2)

    let implMethod2 x y z = 
        x + y + z

    [<Test>]
    let testImpleMethod2 () =
        Assert.AreEqual(implMethod2 1 1 1, 3)

    let implMethod3 x y z r =
        x + y + z + r

    [<Test>]
    let testImpleMethod3 () =
        Assert.AreEqual(implMethod3 1 1 1 1, -1)

    let implMethod4 x y z r s =
        x + y + z + r + s 

    [<Test>]
    let testImpleMethod4 () =
        Assert.AreEqual(implMethod4 1 1 1 1 1, 5)

    let visibleMethod x =
        implMethod1 x x
        + implMethod2 x x x
        + implMethod3 x x x x

TestProxy.fs (implementation of our "Approach")

module TestProxy

open NUnit.Framework

[<Test>]
let run () =
    ///we only want static (i.e. let bound functions of a module), 
    ///non-public methods (exclude any public methods, including this method, 
    ///since those will not be skipped by nunit)
    let bindingFlags = System.Reflection.BindingFlags.Static ||| System.Reflection.BindingFlags.NonPublic

    ///returns true if the given obj is of type TestAttribute, the attribute used for marking nunit test methods
    let isTestAttr (attr:obj) =
        match attr with
        | :? NUnit.Framework.TestAttribute -> true
        | _ -> false

    let assm = System.Reflection.Assembly.GetExecutingAssembly()
    let tys = assm.GetTypes()
    let mutable count = 0
    for ty in tys do
        let methods = ty.GetMethods(bindingFlags)
        for mi in methods do
            let attrs = mi.GetCustomAttributes(false)
            if attrs |> Array.exists isTestAttr then
                //using stdout w/ flush instead of printf to ensure messages printed to screen in sequence
                stdout.Write(sprintf "running test `%s`..." mi.Name)
                stdout.Flush()
                mi.Invoke(null,null) |> ignore
                stdout.WriteLine("passed")
                count <- count + 1
    stdout.WriteLine(sprintf "All %i tests passed." count)

Example Output (using TestDriven.NET)

Notice we never get to testImplMethod4 since it fails on testImpleMethod3:

running test `testImpleMethod1`...passed
running test `testImpleMethod2`...passed
running test `testImpleMethod3`...Test 'TestProxy.run' failed: System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation.
  ----> NUnit.Framework.AssertionException :   Expected: 4
  But was:  -1
    at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner)
    at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeType typeOwner)
    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
    C:\Users\Stephen\Documents\Visual Studio 2010\Projects\FsOverflow\FsOverflow\TestProxy.fs(29,0): at TestProxy.run()
    --AssertionException
    C:\Users\Stephen\Documents\Visual Studio 2010\Projects\FsOverflow\FsOverflow\Program.fs(25,0): at MyNs.Program.testImpleMethod3()

0 passed, 1 failed, 4 skipped (see 'Task List'), took 0.41 seconds (NUnit 2.5.10).