Correct Way To Run Test Set Up with Nunit TestCaseSource

Question

I am trying to run multiple tests using TestCaseSource in NUnit. But I am struggling to get the [SetUp] to run when I want it.

At the moment it works how I want it to but it doesnt feel "right". So the following is the main test case code (simplified):

public class ImportTestCases
{

    ImportTestCases()
    {
        TestData.RunTestSetup();
    }

    public static IEnumerable TestCases
    {
        get
        {
            //run the funciton under test...
            var results = RunFunctionSubjectToTest(TestData.ImportantVar);

            //get multiple results...
            var allProperties =new TestCaseData(o).Returns(true)
                ExpandNestedProperties(results.AllProperties)
                    .ToList()
                    .ConvertAll(o => new TestCaseData(o).Returns(true));

            return allProperties;
        }
    }


}


[TestFixture]
public class ImportTests
{

    [TestFixtureSetUp]
    public void ImporTestSetup()
    {
        TestData.RunTestSetup();
    }

    [Test, TestCaseSource(typeof(ImportTestCases), nameof(ImportTestCases.TestCases))]
    public bool PropertyTest(UnitTestHelper.PropInfo info)
    {
        return info.DoTheyMatch;
    }

}

The trouble here is that [SetUp] does not run before ImportTestCases "TestCases" property "get" is ran. The Constructor of "ImportTestCases" is not ran either. So in order to ensure "RunTestSetup" is ran before ImportVar is referenced I have to do the following:

public static class TestData
{
    private static bool HasSetUpRan = false;
    private static int _importantVar;
    public static int ImportantVar
    {
        get
        {
            if(!HasSetUpRan)
            {
                RunTestSetup();
            }
            return _importantVar;
        }
    }        
    public static void RunTestSetup()
    {
        if (HasSetUpRan)
        {
            return;
        }
        ///do set up
        //e.g. _importantVar = GenerateId();
        //end
        HasSetUpRan= true;
    }

}

As you can see this ensures that Set up has ran before the variable is returned. Sadly this is the only way I have managed to get it to work so far. Which as I say feels "wrong" and over complicated. Perhaps I am overusing testCases here? or I should use some kind of paramatised testcase (is that possible?).

I have tried to simplify the code above so apologies if it simply doesnt make sense what I am trying to test.

The main point is is there a [Setup] that runs before TestCaseSources are created?

Answer 1

The main point is that test cases will be located at the time the tests are loaded. So the routine with [TestFixtureSetUp] attribute will be executed after the "TestCases" property is called. However you could execute some set-up routine within the static constructor. But in order to call it first you need to put your test data in the same class:

[TestFixture]
public class ImportTests
{
    static ImportTests() 
    {
        //Step 1 
        //run your set-up routine
    }

    //Step 3
    [Test, TestCaseSource(nameof(ImportTests.TestCases))]
    public bool PropertyTest(string s) => string.IsNullOrEmpty(s);

    //Step 2
    public static IEnumerable TestCases => new[] {new TestCaseData("").Returns(true)};
}

Answer 2

Maybe one possible solution can be to make your method which creates the TestSource not static and add a default constructor. In the constructor you can do all the initialisation stuff which is necessary for your test cases. And you can still use your TestFixtureSetUp for other initialisation stuff.

[TestFixture]
public class ImportTests
{
    public ImportTests()
    {
        //inititalize test case source
    }

    [TestFixtureSetUp]
    public void ImporTestSetup()
    {
        //inititalize rest of test
    }

    public IEnumerable<string> Fields()
    {
        return new[] { "foo", "bar", "foobar" };
    }

    [Test]
    [TestCaseSource("Fields")]
    public void PropertyTest(string info)
    {
       // Assert
    }
}

Answer 3

Make Test method parameter as Func<YOURParamater>. So when it's time for Nunit to get list of TestCases it return Func not actual parameter. in Test Run time Invoke this Func and get your parameter.

Wrap is just a function which return lambda as Func. bcs lambda cannot be cast to object, contrary to Func.

if you find this answer not satisfactory. please describe reason before rating it down.

public class TestCaseClass
{
    public static IEnumerable Cases
    {
        get
        {

            yield return new TestCaseData(Wrap(() => 1)).SetArgDisplayNames("1");
            yield return new TestCaseData(Wrap(() => 2)).SetArgDisplayNames("2");
        }
    }
    private static Func<T> Wrap<T>(Func<T> fun)
    {
        return fun;
    }
}


[TestCaseSource(typeof(TestCaseClass), nameof(UserTestCases.Cases))]
public bool Tests(Func<int> getInt)
{
    return getInt() == 1;
}

Answer 4

What you are trying to do basically ignores how NUnit works. Consider this sequence:

1. You run a GUI and load a test assembly.
2. In order to give the gui a list of tests, NUnit executes your TestCaseSource method.
3. Your testcase source method determines how many tests there will be and what parameters are passed to each test case.
4. You sit at your desk a long tine looking at the names of all the tests, or maybe posting on twitter. Let's say 20 minutes.
5. You run all your tests, leading to SetUp, TearDown and the test method itself executing.
6. You go to lunch.
7. After lunch, you decide to run the tests again. All the same SetUp, TestMethod, TearDown stuff executes once again. So long as you have not re-compiled an reloaded the test assembly, the testcase source is not used again.

You have to be aware of this sequence, which I have exaggerated for effect, when deciding what to do in your testcase source. Generally, creating long-lived objects is probably not what you want to do.

A few more notes:

• In NUnit, test cases are almost always parameterized. If the test is not parameterized, then it doesn't need a source but can just be a simple test and can do all its own initialization.

• Marking the same method as a test and a source as someone suggested is a really big mistake - pathological. The only reason we don't catch it as an error is that hardly anyone ever tries to do it, since the purposes of a test method and a test case source method are completely distinct.

Your best bet is to use the source to return parameters that can be used by the test to instantiate the objects you need.