Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to Re-run failed JUnit tests immediately?

Is there a way to have an JUnit Rule or something similar that gives every failing test a second chance, just by trying to run it once again.

Background: I have a large Set of Selenium2-WebDriver tests written with JUnit. Due to a very aggressive timing (only short wait periods after the clicks) some tests (1 out of 100, and always a different one) can fail because the server sometimes responds a bit slower. But I can not make the wait period so long that it is definitely long enough, because then the tests will take for ever.) -- So I think it is acceptable for this use case that a test is green even if it needs a second try.

Of course it would be better to have a 2 out of 3 majority (repeat a failing test 3 times, and take them as correct, if two of the tests are correct), but this would be a future improvement.

like image 493
Ralph Avatar asked Nov 28 '11 11:11

Ralph


People also ask

How do you rerun Failed tests with JUnit?

Rerun Failed Tests with JUnit 4 ExampleWe need two classes, one of them is our Rule Class's RetryRule and the other is our Test Class's RetryRuleTest. In RetryRuleTest class, I will open www.swtestacademy.com and get its title and check it with the WRONG expected title.

How do you rerun Failed test cases in JUnit 5?

If you happen to be running your tests using the Gradle build tool, you can use the Test Retry Gradle plugin. This will rerun each failed test a certain number of times, with the option of failing the build if too many failures have occurred overall.

How do you rerun Failed test cases?

To rerun failed test runs automatically during the test run itself, we implement IRetryAnalyzer interface provided by TestNG. By overriding retry() method of the interface in your class, you can control the number of attempts to rerun a failed test case.

How do you repeat a test in JUnit?

@RepeatedTest annotation is used to repeat a test for a specific number of times. The basic syntax of @RepeatedTest is as follows: The annotation @RepeatedTest is used for a test method instead of the annotation @Test. A number is passed as the input parameter of the annotation.


1 Answers

You can do this with a TestRule. This will give you the flexibility you need. A TestRule allows you to insert logic around the test, so you would implement the retry loop:

public class RetryTest {     public class Retry implements TestRule {         private int retryCount;          public Retry(int retryCount) {             this.retryCount = retryCount;         }          public Statement apply(Statement base, Description description) {             return statement(base, description);         }          private Statement statement(final Statement base, final Description description) {             return new Statement() {                 @Override                 public void evaluate() throws Throwable {                     Throwable caughtThrowable = null;                      // implement retry logic here                     for (int i = 0; i < retryCount; i++) {                         try {                             base.evaluate();                             return;                         } catch (Throwable t) {                             caughtThrowable = t;                             System.err.println(description.getDisplayName() + ": run " + (i+1) + " failed");                         }                     }                     System.err.println(description.getDisplayName() + ": giving up after " + retryCount + " failures");                     throw caughtThrowable;                 }             };         }     }      @Rule     public Retry retry = new Retry(3);      @Test     public void test1() {     }      @Test     public void test2() {         Object o = null;         o.equals("foo");     } } 

The heart of a TestRule is the base.evaluate(), which calls your test method. So around this call you put a retry loop. If an exception is thrown in your test method (an assertion failure is actually an AssertionError), then the test has failed, and you'll retry.

There is one other thing that may be of use. You may only want to apply this retry logic to a set of tests, in which case you can add into the Retry class above a test for a particular annotation on the method. Description contains a list of annotations for the method. For more information about this, see my answer to How to run some code before each JUnit @Test method individually, without using @RunWith nor AOP?.

Using a custom TestRunner

This is the suggestion of CKuck, you can define your own Runner. You need to extend BlockJUnit4ClassRunner and override runChild(). For more information see my answer to How to define JUnit method rule in a suite?. This answer details how to define how to run code for every method in a Suite, for which you have to define your own Runner.

like image 195
Matthew Farwell Avatar answered Sep 24 '22 18:09

Matthew Farwell