Stopping JUnit suite if particular test fails

Question

I have a JUnit test suite in the form:

@RunWith(Suite.class)
@Suite.SuiteClasses( { xx.class, yy.cass })

public class AllTests {

public static Test suite() {
    TestSuite suite = new TestSuite(AllTests.class.getName());
    //$JUnit-BEGIN$

    //$JUnit-END$
    return suite;
}
}

This then calls vanilla tests like this:

public class xxx {

@Test
public void test () throws {
    ...

I have a situation where I'd like to stop the rest of the test suite running if there's an error or fail in the first test. But errors / fails in the others are ok and the suite should complete as many other tests as it can. Basically the first test failing would indicate it isn't safe to run the rest.

Is this possible?

Answer 1

First you need junit RunListener:

import org.junit.runner.notification.Failure;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;

public class FailureListener extends RunListener {

    private RunNotifier runNotifier;

    public FailureListener(RunNotifier runNotifier) {
        super();
        this.runNotifier=runNotifier;
    }

    @Override
    public void testFailure(Failure failure) throws Exception {
        super.testFailure(failure);
        this.runNotifier.pleaseStop();
    }
}

Then prepare a suite:

public class StopOnFailureSuite extends Suite {

    public StopOnFailureSuite(Class<?> klass, Class<?>[] suiteClasses) throws InitializationError {
        super(klass, suiteClasses);
    }

    public StopOnFailureSuite(Class<?> klass) throws InitializationError {
        super(klass, klass.getAnnotation(SuiteClasses.class).value());
    }

    @Override
    public void run(RunNotifier runNotifier) {
        runNotifier.addListener(new FailureListener(runNotifier));
        super.run(runNotifier);
    }
}

And run your suite:

@RunWith(StopOnFailureSuite.class)
@Suite.SuiteClasses({
    FirstTestClass.class,
    SecondTestClass.class,
    ...
})

Answer 2

What's wrong with calling System.exit()?

Answer 3

If it's first test then consider moving its validation to @BeforeClass and throw exception if it fails. Then only @AfterClass method would run in case of this exception.

Of course, that way you lack all the fixture artifacts created in test setup method(s).