Has JUnit4 begun supporting ordering of test? Is it intentional?

Question

A newbie to JUnit (in fact JUnit 4) and came across suite way of executing test

@RunWith(Suite.class)
@Suite.SuiteClasses(
        {                               
                CreateNewProfile.class,
                EditProfile.class,

        })
public class ProfileTestSuite {

}

This is the code sample I came across while browsing through the test code base at my new employer. During execution I fund that - CreateNewProfile tests are executed first and then EditProfile, which does make sense but then it introduces dependency among tests.

I have been following non dependent test mechanism from couple of months (though I used to use TestNG and not JUnit) and would expect EditProfile being able to be executed in isolation as well. That is edit profile should take care of creating profile and then editing it and then asserting the operations.

My question here is - has Junit 4 introduced test ordering feature. Is this feature intended or one easter egg as I always felt JUnit = independent tests.

Answer 1

No JUnit does not support test ordering, except in the manner that you say, through Suite. This only defines the order in which the test classes are executed. This has been there for a long time, including JUnit 3 Suite classes.

For a more complete explanation, there are three things we need to talk about here:

1. the ordering of tests classes within a test suite
2. the ordering of test classes when they are found by reflection by Eclipse or Maven
3. the ordering of test methods (annotated with @Test) within a test class.

The ordering of tests classes within a test suite

When you specify the list of classes to be executed in a test suite, you're defining an array, and these test classes will be executed in order, except when you're doing parallel execution. Unfortunately, this allows the introduction of dependencies between your test classes.

The ordering of test classes when they are found by reflection

When searching the classpath for classes, the order in which they are found is not guaranteed, so cannot be depended upon. It isn't actually JUnit which is doing the searching, but the Eclipse Junit plugin, or maven surefire or failsafe.

The ordering of test methods within a test class

JUnit does not guarantee the order of execution of tests within a class. Most of the time, on most JVMs pre version 7, the order in which they are found using reflection is in declaration order, ie the order they are in the file. This is the order in which they are executed. However, with JVM 7, this is no longer guaranteed, so there won't be a consistent order. There is a github issue #293 Sort test methods for predictability open with suggested solutions, and there is a thread on the junit mailing list: Alphabetizing test method run order?. So you cannot depend upon the order that tests will be executed using JUnit, but this is currently under discussion.