How can I make JUnit 4.8 run code after a failed test, but before any @After methods?

Question

I'm driving a suite of Selenium tests (actually WebDriver-backed Selenium) using JUnit 4.8.2. I'd like the tests to automatically take a screenshot of the browser as soon as the test fails an assertion. All the tests inherit from SeleniumBaseTestCase, and the majority then further inherit from from SeleniumBastTestCaseWithCompany (which uses @Before and @After methods to create and then clean up common test data via Selenium).

I've tried adding a subclass of TestWatchman as a @Rule in SeleniumBaseTestCase, overriding TestWatchman's failed method to take the screenshot. The trouble is that the @After methods cleaning up the test data are being run before TestWatchman's failed method is called, so the screenshots are all of the final step of the clean-up, not the test that failed.

Looking into it a little, it seems that TestWatchman's apply method just calls the passed Statement's evaluate method (the only exposed method), which calls the @After methods, leaving TestWatchman (or any other Rule) no chance to insert any code between the execution of the test and of the @After methods, as far as I can tell.

I've also seen approaches that create a custom Runner to alter the Statements created so that methods annotated with the custom @AfterFailure are run before @After methods (so the screenshot can be taken in such an @AfterFailure method), but this relies on overriding BlockJUnit4ClassRunner's withAfters method, which is deprecated and due to become private, according to the documentation, which suggests using Rules instead.

I've found another answer on SO about the @Rule lifecycle that makes it sound like this simply might not be possible in JUnit 4.8, but may be possible in JUnit 4.10. If that's correct then fair enough, I'd just like confirmation of that first.

Any thoughts on an elegant and future-proof way in which I can achieve what I want would be much appreciated!

Answer 1

You are right in your analysis, @Befores and @Afters are added to the list of Statements before any Rules. The @Before gets executed after the @Rule and the @After gets executed before the @Rule. How you fix this depends on how flexible you can be with SeleniumBaseTestCaseWithCompany.

The easiest way would be to remove your @Before/@After methods and replace them with an ExternalResource. This could look something like:

public class BeforeAfterTest {
    @Rule public TestRule rule = new ExternalResource() {
        protected void before() throws Throwable { System.out.println("externalResource before"); }
        protected void after() { System.out.println("externalResource after"); }
    };

    @Test public void testHere() { System.out.println("testHere"); }
}

this gives:

externalResource before
testHere
externalResource after

This field can be put into your base class, so it gets inherited/overridden. Your problem with ordering between @After and your rules then goes away, because you can order your rules how you like, using @RuleChain (in 4.10, not 4.8).

If you can't change SeleniumBaseTestCaseWithCompany, then you can extend BlockJUnit4ClassRunner, but don't override withAfters, but override BlockJUnit4ClassRunner#methodBlock(). You can then call super.methodBlock, and reorder the Statements as necessary[*].

[*]You could just copy the code, and reorder the lines, but withRules is private and therefore not callable from a subclass.