Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I make my JUnit tests run in random order?

Tags:

I have the classical structure for tests, I have a test suite of different suites like DatabaseTests, UnitTests etc. Sometimes those suites contains other suites like SlowDatabaseTests, FastDatabaseTests etc.

What I want is to randomize the running order of tests so I will make sure they are not dependent to each other. Randomization should be at every level, like suite should shuffle test class order, and test class should shuffle test method order.

If it is possible to do this in Eclipse that will be the best.

like image 216
nimcap Avatar asked Sep 18 '09 12:09

nimcap


People also ask

Do JUnit tests run sequentially?

Basically, JUnit doesn't support ordering and the tests should be able to be run in any order. If you really need to have dependencies between tests, use TestNG, where you can have dependencies.

Can we set priority in JUnit?

In general, you can't specify the order that separate unit tests run in (though you could specify priorities in TestNG and have a different priority for each test). However, unit tests should be able to be run in isolation, so the order of the tests should not matter.


2 Answers

You do have a Sortable but I can't see how you would use it.

You could extend BlockJUnit4ClassRunner and have computeTestMethods() return a randomized copy of super.computeTestMethods(). Then use the @RunWith to set that as the runner to use.

e.g.

package com.stackoverflow.mlk;  import java.util.Collections;  import org.junit.runners.BlockJUnit4ClassRunner; import org.junit.runners.model.InitializationError;  public class RandomBlockJUnit4ClassRunner extends BlockJUnit4ClassRunner {      public RandomBlockJUnit4ClassRunner(Class<?> klass)             throws InitializationError {         super(klass);     }      protected java.util.List<org.junit.runners.model.FrameworkMethod> computeTestMethods() {         java.util.List<org.junit.runners.model.FrameworkMethod> methods = super.computeTestMethods();         Collections.shuffle(methods);         return methods;     }  } 

Then

@RunWith(com.stackoverflow.mlk.RandomBlockJUnit4ClassRunner.class) public class RandomOrder {     @Test     public void one() {     }      @Test     public void two() {     }      @Test     public void three() {     } } 
like image 127
Michael Lloyd Lee mlk Avatar answered Oct 17 '22 12:10

Michael Lloyd Lee mlk


https://github.com/KentBeck/junit/pull/386 introduces some orders but not RANDOM. Probably you do not really want this; tests should run deterministically. If you need to verify that different permutations of tests still pass, either test all permutations; or, if this would be impractically slow, introduce a “random” seed for shuffling that is determined by an environment variable or the like, so that you can reproduce any failures. http://hg.netbeans.org/main/file/66d9fb12e98f/nbjunit/src/org/netbeans/junit/MethodOrder.java gives an example of doing this for JUnit 3.

like image 42
Jesse Glick Avatar answered Oct 17 '22 14:10

Jesse Glick