Do not run a category of slow JUnit tests by default in Maven and every IDE (Eclipse, IntelliJ, ...) without an explicit TestSuite

Question

I have a set of really slow tests, which take a week to run. (They literally run some code non-stop for about a week).

Naturally, no developer (or even the default build job) wants to run these tests. Only a specific, separate build job has the time to run them. So these tests needs to be disabled by default.

JUnit's categories seemed perfect for this: I annotated those slow tests with @Category(SlowTests.class). Problem is that they are still run because:

• I don't want to maintain TestSuite classes because we add Tests on a daily basis. I have 0 TestSuite classes: Maven, IntelliJ and Eclipse just run all the Test classes from my module. Including all those slow tests.
• Even if we had TestSuite classes, most of the open source contributors would still try to run all Test classes from IntelliJ or Eclipse, think it hangs and give up running tests. Those slow tests needs to be excluded by default. Preferably without any extra configuration in IntelliJ or Eclipse.

How do I exclude a category of slow JUnit tests by default without using an explicit TestSuite?

Answer 1

This works by default, in Maven, IntelliJ and Eclipse:

import static org.junit.Assume.assumeTrue;

@Test
public void mySlowTest() {
    assumeTrue("true".equals(System.getProperty("runSlowTests")));
    ...
}

To run them anyway, simply add VM argument -DrunSlowTests=true.

Semantically speaking, it's totally wrong. But it works :)

Answer 2

As far as I know there is no way of preventing Eclipse from running certain tests by default.

Running certain categories from Maven is easy enough using

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12.4</version>
    <configuration>
        <excludedGroups>${tests.exclude}</excludedGroups>
    </configuration>
</plugin>

And then define tests.exclude in certain maven profiles.

Maintaining test suites in JUnit is indeed too much work with the current version of JUnit as I've written about in a blogpost. I also explain how a library called cpsuite automatically does the Suite administration for you like this:

@RunWith(ClasspathSuite.class) // Loads all unit tests it finds on the classpath
@ExcludeBaseTypeFilter(SlowTest.class) // Excludes tests that inherit SlowTest
public class FastTests {}

However, in both methods, Eclipse by default will still just run all Java files with a @Test annotation in them.