How to name and organize unit tests that test a method with multiple parameters?

Question

Given this method that has to be tested:

// Search the given value using the provided search options
// Return true if the searchValue was found; false otherwise 
bool Search(string searchValue, bool useExactSearch, bool useIndexing)

I have 6 significant searchValues (one with ponctuation, one with accented characters, one with line breaks, etc) that I need to validate with each possible combination of the useExactSearch and useIndexing. This means 54 test cases.

How do you go about that? Do you really write 54 unit tests? If so, how do you name them? Do you write tests only for the most significant cases? Do you write a single unit tests that loops over a table of param values and expected results? If I do a single unit test, it's harder to find which case is broken when the Continuous Integration reports a failure.

Answer 1

If you're using NUnit (.Net), you can do this:

[TestCase("John*",true, false, false)]
[TestCase("*user*",false, true, true)]
[TestCase(".",true, false, true)]
public void SearchTest(string param1, bool param2, bool param3, bool expectedResult)
{
    YourClass clazz = new YourClass();
    bool result = clazz.Search(param1, param2, param3);
    Assert.AreEqual(expectedResult, result);
}

The NUnit Test Runner will execute this 3 times. And, as you can see here, the report shows all test cases separated, which helps you identify who broke the build. This is probably available on most xUnit frameworks.

Answer 2

In his book The Art of Unit Testing Roy Osherove recommends using this naming convention:

MethodUnderTest_ConditionUnderTest_ExpectedBehaviour()

Which seems reasonable. As to how to do multiple tests of the same function with different params, I think individual tests will make it clearer what went wrong, and you can used refactoring to minimise the duplication between the tests.

On a side note, it's not language agnostic, but NUnit allows for parameterised tests, so other unit testing frameworks might do this too. That's a nice halfway house between 'one single difficult to understand the reason for failures' test and 54 'almost identical, with loads of duplication' tests.