I have several test classes, each with several test methods. All tests use the same test database in the background. Each test class initializes its database contents and then tests stuff in several test methods.
When I run each test individually, they all pass. But when I run several tests at the same time (either using maven or my IDE, IntelliJ), the methods of different test classes are run interleaved, eg. the database initialization of the second class runs after the first class has started but before all test methods of first class have been run so these methods will fail (because the database already contains second class's data).
The simplest solution would be to force the TestNG runner to run the classes in succession (ie. wait for all the test methods of a test class to finish before running test methods of another class). Can this be done?
I can probably do this by specifying each class as a separate test in my suite, but I don't want to do this as this means I'd have to add something to the suite whenever I add a test class which is clumsy and error-prone.
Simply asking TestNG to not parallelize anything (eg. setting thread count to 1 or disabling parallel running) doesn't help here since methods still get run in the wrong order (though not simultaneously).
One option would be to use a different database for each test class, but I don't see a simple way to do this (using JPA and Guice).
I'm not currently using DBUnit, Unitils etc.; I don't know these tools very well but I got the impression the don't solve my problems.
I'm using JPA to initialize database in each test class (ie. create entity objects and presist them).
Even in sequential mode TestNG could interleave test methods from the same suite. It does guarantee the sequence @BeforeClass -> @Test -> @AfterClass but it can do something like:
before class1
test class1.method1
before class2
test class2.method1
test class1.method2
after class1
test class2.method2
after class2
The solution is to force each class in a different suite (which are executed truly sequentially). As of version 2.16, the maven surefire plugin puts each class in a separate suite so the problem is fixed.
On the other hand, IDEA (even the latest 13 EAP) generates an xml file with all classes in the same suite. Hopefully IDEA will follow suit and fix this too. Interleaved tests are a showstopper when working with shared resources such as databases.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With