If I @Ignore a test class in JUnit4, does @BeforeClass still run?

Question

Quick background: I've been hunting down a Maven / Surefire test-running problem for days now, and I've narrowed it down to a small number suspect of tests. The behavior I'm seeing is insane. I start with mvn clean test: 250 tests run, 0 skipped. Now, I move the suspect test into src/test/java and try again: 146 tests run, 0 skipped! The output of Maven gives no clue that other tests aren't being run, even with the -X flag.

That brings me to my question: the reason I call the test 'suspect' is that the whole class is decorated with @Ignore, so I would imagine that including it in my test sources should have no effect at all. Then it occurred to me -- those classes have @BeforeClass/@AfterClass methods that manage a dummy Zookeeper server. It's resulted in wonky behavior before, which is why we have the tests @Ignored.

If JUnit is running the before/after code but ignoring the tests, I have no idea what might happen (but it'd probably be super bad). Is this happening? Is this supposed to happen? If so, how am I supposed to say "for reference, here's a test that should work but needs fixing" when it includes @BeforeClass / @AfterClass? Also of substantial interest: what the hell is this doing to Surefire / Maven, that it causes unrelated tests to fall off the face of the Earth?

Answer 1

If you have a test with the @Ignore annotation, then it is normal behaviour for the @BeforeClass & @AfterClass to get run, whether or not all of the tests are @Ignored.

If, however, the Class has an @Ignore annotation, then the @BeforeClass & @AfterClass don't get run.

For maven, if you don't want to run any tests in a particular class, then you have to ignore them in surefire or failsafe. Add this to the maven configuration (see Maven Surefire Plugin)

<excludes>
 <exclude>**/FoobarTest.class</exclude>
</excludes>

Answer 2

Environment: JDK 1.6, surefire plugin 2.9, jUnit 4.8.1, Maven 3.0, 3.0.3, 2.2.1.

I created this test class:

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Test;

@Ignore
public class IgnoreTest {

    @BeforeClass
    public static void beforeClass() {
        System.out.println("BEFORE CLASS");
    }

    @AfterClass
    public static void afterClass() {
        System.out.println("AFTER CLASS");
    }

    @Test
    public void test1() throws Exception {
        System.out.println("test1");
    }

    @Test
    public void test2() throws Exception {
        System.out.println("test2");
    }

    @Test
    public void test3() throws Exception {
        System.out.println("test3");
    }
}

Then mvn clean test print this:

Running hu.palacsint.stackoverflow.q7535177.IgnoreTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.015 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 1

Works as you expected. If I remove the @Ignore and run mvn clean test again it prints this:

Running hu.palacsint.stackoverflow.q7535177.IgnoreTest
BEFORE CLASS
test2
test1
test3
AFTER CLASS
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec

Results :

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

So, it works for me with three different Maven versions. No @BeforeClass/@AfterClass was run in @Ignored classes.

There is one (maybe more) situation when @BeforeClass/@AfterClass methods could run in an @Ignored test class. It's when your ignored class has a not ignored subclass:

import org.junit.Test;

public class IgnoreSubTest extends IgnoreTest {

    @Test
    public void test4() throws Exception {
        System.out.println("test4 subclass");
    }

}

Results of mvn clean test:

Running hu.palacsint.stackoverflow.q7535177.IgnoreTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.047 sec
Running hu.palacsint.stackoverflow.q7535177.IgnoreSubTest
BEFORE CLASS
test4 subclass
test1
test2
test3
AFTER CLASS
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 sec

Results :

Tests run: 5, Failures: 0, Errors: 0, Skipped: 1

In this case the @BeforeClass and the @AfterClass methods runs because they are methods of the IgnoreSubTest test class.