What's the difference between [Test] (optional [Values]) versus [TestCase] (optional ExpectedResult)?

Question

The actual question is at the bottom. I've tried to make the post as short as possible already and I hope that I've managed to straighten most of the unclear parts of the issue.

According to the docs, we have in nUnit (among other attributes)

• Test
• Values
• TestCase

I've been reading in the docs and using all all the above since long time so I'm well familiar with how they behave. However, one thought has always bugged me and by now, I'm embarrassed to ask.

public int Output(int input) { return input + 1; } 

The way we could set up test for a simple method like the one above, I can think of four approaches. I've seen most of them pretty much everywhere but I can't really tell if those are different flavors leading to the same under-the-hood functionality or if there's a technical difference that I'm simply not aware of.

For instance - perhaps #2 can be parallellized whereas the other can't (it's just a dummy example to show that there might be magic that we're not aware of).

Test attribute only

[Test]
public void OutputIsWorking()
{
  List<int> inputs = new List<int>{ 1, 2, 3 };
  for(int i = 0; i < inputs.Count; i++)
    Assert.That(inputs[i] + 1, Is.EqualTo(Output(input)));
}

Test and Values attributes combined

[Test]
public void OutputIsGreat([Values(1,2,3)] int input)
{
  int output = Output(input);
  Assert.That(input + 1, Is.EqualTo(output));
}

TestCase attribute only

[TestCase(1, 2)]
[TestCase(2, 3)]
[TestCase(3, 4)]
public void DivideTest(int input, int expectation)
{
  int output = Output(input);
  Assert.That(output, Is.EqualTo(expectation));
}

TestCase and ExpectedResult attributes combined

[TestCase(1, ExpectedResult = 2)]
[TestCase(2, ExpectedResult = 3)]
[TestCase(3, ExpectedResult = 4)]
public void DivideTest(int input)
{
  return Output(input);
}

So, the question is if those approaches differ on a technical level or if it's just whatever the code is most fond of at the moment. We prefer to lower the number of choices a developer needs to make and providing such a wide range of equivalents seems redundant and confusing.

If they differ, then I'd like to know how. I haven't found that in the docs. If they don't differ, then I'm curious why. Are we talking about "let people decide what they like" or rather "this is a legacy thing that can't be removed"?

Answer 1

There are some subtle differences.

• Test cases count

  In the first example (Test attribute only), you're writing a single test case. If the test fails on input 2, the whole test will fail, and input 3 will not be tested at all.

  In all the other examples, you're writing separate test cases, so even if 2 fails, NUnit will still test all the remaining cases, and show then as such in the test report.

  You can also opt-in to parallel test execution on NUnit 3 when you have several test cases. If you use a different runner such as NCrunch you can also get parallelism easily by using separate test cases.

• Combinatorial

  When you're using ValuesAttribute, and you have several parameters, NUnit will execute all possible combinations of input values as separate tests. By using TestCaseAttribute, you'd have to write all the inputs yourself.

  You can use CombinatorialAttribute to mark this explicitly, or use PairwiseAttribute or SequentialAttribute for a different approach.

Other differences are mostly just a matter of preference.

Using ExpectedResult frees you from having to write an assert (it's implicitly inserted by NUnit for you), and having to write less code is arguably a good thing. I wouldn't write the "TestCase attribute only" example myself, I'd rather use the ExpectedResult feature for this, as I think it's more readable this way (the output parameter stands out).

I'd favor TestCase over Values when I do not want the combinatorial behavior, but it's just a matter of style when you have a single parameter. With several input parameters, you'll probably need both approaches anyway.