Is there a way to make Eclipse run a JUnit test multiple times until failure?

Question

We occasionally have bugs that appear once in every X runs. Before people check in stuff (where it is automatically JUnit'd), our devs need to pass JUnit locally via Eclipse.

Is there some convenient way (built in or high-quality Plugin) to make Eclipse run the same test X times and stop if there's a failure? An alternative to just clicking Run X times?

Note that I'm looking for something in the UI (e.g., right click and say "Run X times" instead of just "Run").

Answer 1

If the for loop works, then I agree with nos.

If you need to repeat the entire setup-test-teardown, then you can use a TestSuite:

1. Right-click on the package containing the test to repeat
2. Go to New and choose to create a JUnit test SUITE
3. Make sure that only the test you want to repeat is selected and click through to finish.
4. Edit the file to run it multiple times.

In the file you just find the

addTestSuite(YourTestClass.class) 

line, and wrap that in a for loop.

I'm pretty sure that you can use addTest instead of addTestSuite to get it to only run one test from that class if you just want to repeat a single test method.

Answer 2

If you really want to run a test class until failure, you need your own runner.

@RunWith(RunUntilFailure.class) public class YourClass {      // ....  } 

which could be implemented as follows...

package com.example;  import org.junit.internal.runners.*; import org.junit.runner.notification.*; import org.junit.runner.*;  public class RunUntilFailure extends Runner {      private TestClassRunner runner;      public RunUntilFailure(Class<?> klass) throws InitializationError {         this.runner = new TestClassRunner(klass);     }      @Override     public Description getDescription() {         Description description = Description.createSuiteDescription("Run until failure");         description.addChild(runner.getDescription());         return description;     }      @Override     public void run(RunNotifier notifier) {         class L extends RunListener {             boolean fail = false;             public void testFailure(Failure failure) throws Exception { fail = true; }         }         L listener = new L();         notifier.addListener(listener);         while (!listener.fail) runner.run(notifier);     }  } 

...releasing untested code, feeling TDD guilt :)