Mark unit test as an expected failure in JUnit

Question

How can I mark a test as an expected failure in JUnit 4?

In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an @expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.

Here's what my current test looks like:

@Test public void unmarshalledDocumentHasExpectedValue()  {     doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));     final ST title = doc.getTitle();     assertThat(doc.getTitle().toStringContent(), equalTo("Expected")); } 

That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think @Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.

In Python I can use the expectedFailure decorator:

class ExpectedFailureTestCase(unittest.TestCase):     @unittest.expectedFailure     def test_fail(self):         self.assertEqual(1, 0, "broken") 

With Qt's QTestLib in C++, you can use QEXPECT_FAIL:

QEXPECT_FAIL("", "Will be fixed next version", Continue); QCOMPARE(i, 42); 

In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?

Answer 1

I'm not quite getting the specifics of your scenario, but here's how I generally test for expected failure:

The slick new way:

@Test(expected=NullPointerException.class) public void expectedFailure() {     Object o = null;     o.toString(); } 

for older versions of JUnit:

public void testExpectedFailure() {     try {         Object o = null;         o.toString();         fail("shouldn't get here");     }     catch (NullPointerException e) {         // expected     } } 

If you have a bunch of things that you want to ensure throw an exception, you may also want to use this second technique inside a loop rather than creating a separate test method for each case. If you were just to loop through a bunch of cases in a single method using expected, the first one to throw an exception would end the test, and the subsequent cases wouldn't get checked.