APPROACH#1: Login before each test:
PROS:
CONS:
APRROACH#2: Login once and run all tests:
PROS:
CONS:
Although both the approaches have their pros and cons, I'm still not sure which is the best way to go. Is it really a waste of resources to login before each test as a separate session? There is no DB connection involved. The only potential problem I see with APPROACH#1 is that since every test is a Java process, there is a chance of overloading the system with too many simultaneous java processes.
The answer is, You decide.
But, here is my opinion:
@BeforeSuite is too risky and not realistic as the UI tests are the most flakiest in testing world. There are thousand different reasons the login process can fail. Specially if you run those tests automatically from CI server or through some other automated process your full test suit will not execute.
@BeforeTest is too redundant I believe even though sometimes you need to just to refresh the environment and go back to the known state. Except for those scenario I will not do this either.
Upto this point and as per my knowledge @BeforeClass is the best approach. Since, this will run just once per test class and log in once and execute all the tests. You can use @BeforeTest in conjunction to this just to reset the environment to a known state so that other tests know where to start from
If "a waste of resources" is your cause for concern, explore reducing the number of integration tests you need to run (e.g. by creating more unit / functional tests). You can configure the number of threads TestNG uses, and how parallel your tests are, so the extra sessions shouldn't crash your test process.
Integration tests are supposed to be expensive in order to actually model the whole system. If users need to log in, your test should be logging in. If you try to fake or modify the behavior for testing you only make your tests less useful; and seeing as they're expensive, you want to make them as useful as possible. You also always want to make your tests independent of each other where possible; once one test fails, having thirty other tests fail for the same reason is just noise that makes reporting and responding to failure more difficult.
Therefore what you should do is identify tests that are sure to fail if other tests fail (e.g. all tests that need to log in will fail if your login test fails) and avoid running them. You can do this with dependencies. I would suggest defining groups for your entry-point tests, like logging in, and using dependsOnGroups
to ensure all tests that have to log in only run once that's verified.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With