How can I build a configurable JUnit4 test suite?

Question

Guava has an extensive set of tests for collection implementations written in JUnit3 that look like:

/*
 * Copyright (C) 2008 The Guava Authors
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
public class CollectionRemoveTester<E> extends AbstractTester<E> {

  @CollectionFeature.Require(SUPPORTS_REMOVE)
  @CollectionSize.Require(absent = ZERO)
  public void testRemove_present() {
     ...
  }
}

and then different collections are tested by using TestSuiteBuilders that pass in a set of features and generators for the collection type, and a heavily reflective framework identifies the set of test methods to run.

I would like to build something similar in JUnit4, but it's not clear to me how to go about it: building my own Runner? Theories? My best guess so far is to write something like

abstract class AbstractCollectionTest<E> {
   abstract Collection<E> create(E... elements);
   abstract Set<Feature> features();

   @Test
   public void removePresentValue() {
      Assume.assumeTrue(features().contains(SUPPORTS_REMOVE));
      ...
   }
}

@RunWith(JUnit4.class)
class MyListImplTest<E> extends AbstractCollectionTest<E> {
  // fill in abstract methods
}

The general question is something like: how, in JUnit4, might I build a suite of tests for an interface type, and then apply those tests to individual implementations?

Answer 1

In Junit you can use categories. For example this suite will execute al test from the AllTestSuite annotated as integration:

import org.junit.experimental.categories.Categories;
import org.junit.experimental.categories.Categories.IncludeCategory;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;

@RunWith(Categories.class)
@IncludeCategory(Integration.class)
@Suite.SuiteClasses ({AllTestsSuite.class} )
public class IntegrationTestSuite {}

You can also use @ExcludeCategory. This is usefull to remove slow tests. Categories classes are just plain old Java classes or interfaces. For example:

public interface Integration{}
public interface Performance{}
public interface Slow{}
public interface Database{}

You only need to anotate your tests acordingly:

@Category(Integration.class)
public class MyTest{

   @Test
   public void myTest__expectedResults(){
   [...]

One test might have more than one category like this:

   @Category({Integration.class,Database.class})  
   public class MyDAOTest{

For simplicity I usually create a Suite with all classes in the test folder using google toolbox:

import org.junit.runner.RunWith;

import com.googlecode.junittoolbox.ParallelSuite;
import com.googlecode.junittoolbox.SuiteClasses;

@RunWith(ParallelSuite.class)
@SuiteClasses({"**/**.class",           //All classes
             enter code here  "!**/**Suite.class" })    //Excepts suites
public class AllTestsSuite {}

This works incluiding in AllTestSuite all classes in the same folder and subfolders even if they don't have the _Test sufix. But won't be able to see test that are not in the same folder or subfolders. junit-toolbox is available in Maven with:

<dependency>
    <groupId>com.googlecode.junit-toolbox</groupId>
    <artifactId>junit-toolbox</artifactId>
    <version>2.2</version>
</dependency>

Now you only need to execute the Suite that suits your needs :)

UPDATE: In Spring there is the @IfProfileValue annotation that allows you to execute test conditionally like:

@IfProfileValue(name="test-groups", values={"unit-tests", "integration-tests"})
@Test
public void testProcessWhichRunsForUnitOrIntegrationTestGroups() {

For more information see Spring JUnit Testing Annotations