Testing API which returns multiple values with JUnit

Question

I would like to test an API, which received one argument and returns a set. The test invokes the API with an argument and checks if the returned set contains expected values.

Suppose, I have to test the API with arguments arg1, arg2, and arg3 and check if values a, b, c appear in the returned set. That is, my test case looks as follows:

• invoke the API with arg1 and check if a, b, c appear in the returned set.
• invoke the API with arg2 and check if a, b, c appear in the returned set.
• invoke the API with arg3 and check if a, b, c appear in the returned set.

How to develop this test case with Junit 4 ? What if I have to add arg4 ? What if I have to check if value d appear in the returned set ? Can I read the list of arguments and expected values from the configuration?

Answer 1

Fluent assertions

First of all, use FEST-Assertions library to introduce pleasantly looking assertions with meaningful error messages:

assertThat(method(arg1)).containsExactly(a, b, c);
assertThat(method(arg2)).containsExactly(a, b, c);
assertThat(method(arg3)).containsExactly(a, b, c);

The BDD way

But I understand your question is not about the syntax, but about methodology: what should you do if arg4 needs to be tested? Well, if arg1 through arg4 have a different semantic meaning, I would advice you to have a separate test for each argument. Very verbose, but also extremely readable (pseudocode):

@Test
public void shouldReturnAbcWhenSomeArgumentUsed() {
  //given
  Object arg = arg1;

  //when
  Set<Object> result = method(arg);

  //then
  assertThat(result).containsExactly(a, b, c);
}

..and repeat this for every test. The key part is: method name should represent the meaning of an argument, what does this method actually test, what do you expect, what is the scenario?

Consider testing isEven method. I would recommend the following tests:

• shouldReturnTrueForZero
• shouldReturnTrueForSmallPositiveEvenNumber
• shouldReturnTrueForLargePositiveEvenNumber
• shouldReturnFalseForSmallPositiveOddNumber
• shouldReturnFalseForLargePositiveOddNumber
• ... and mirror the tests for negative numbers

Each test represent a slightly different, well defined scenario. On the other hand you might generate literally thousands of shouldReturnFalseWhen227, but what is the value of such a test suite, except it's large? Test semantically different arguments and corner cases, defining precisesly what case is being tested.

Parameterized way

If you really want to have just a single generic test method, Parameterized runner is the way to go. I think the example is self-explanatory. NB: you can parameterize expected values as well.

@RunWith(value = Parameterized.class)
public class JunitTest6 {

    private Object arg;

    public JunitTest6(Object arg) {
        this.arg = arg;
    }

    @Parameterized.Parameters
    public static Collection<Object[]> data() {
        return Arrays.asList(
                new Object[][]{
                        {arg1},
                        {arg2},
                        {arg3}
                });
    }

    @Test
    public void testMethod() {
        assertThat(method(arg)).containsExcatly(a, b, c);
    }

}

Based on this.

Answer 2

I would usually turn to Hamcrest for something like this -- it's a library for declaratively writing "matchers", and it plays very nicely with JUnit.

However, this question on SO points out that although this can be done with Hamcrest, a simpler way is just to use the containsAll method from java.util.Collection:

ArrayList<Integer> expected = new ArrayList<Integer>();
expected.add(1); expected.add(2); expected.add(3);

assertTrue(actual.containsAll(expected));