How can I analyze a junit test suite?

Question

We have a number of unit tests being run with roboletric.

I'm looking to gather metrics on meta data such as longest tests to run, tests that are putting the thread to sleep, basically anything that can be leveraged to improve test run times.

Answer 1

Your concern is to reduce testing execution time. You can do it in various ways.

At first, you have to make some decisions that which test cases are mandatory, which test cases need to be run in monthly basis etc.

After defining mandatory test cases, you can create a suite using grouping of your test cases. If you run the groups in parallel, it will reduce a huge amount of time. But your grouping preparation need to be tricky so that every group can take similar type of time. That will be the best way.

For JUnit,

you can use parallel process and thread pooling to how much thread will run your test cases(like 20 threads for 5000 tcs)

<build>
    <plugins>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.7.1</version>
        <configuration>
            <parallel>classes</parallel>
            <threadCount>20</threadCount>
        </configuration>
    </plugin>
    </plugins>
</build>

If we calculate here..Suppose, every test case requires 3 sec for execution.

For 5000*3 = 15000sec/3600 = 4.166 hr.

Now, using parallel processing it will take (5000/20)*3 = 750sec/3600 = 0.21hr or `12.5 minutes only.

Resource Link: Running junit tests in parallel in a Maven build?

For TestNG, Appium, Jenkins etc

For UI testing you can run multi-browser so that parallelly they can execute more test cases.

You can use multiple node so that at a time multiple node can run and reduce time.

In some cases, you can be more tricky that you will not login in every test cases. There will be a starting point. Every test case will end and will return that starting point. By using this way, we can also reduce time. But it is violation of F.I.R.S.T.

For analyzing, reporting and showing the status of code coverage, execution time, various types of tools are used. JaCoCo is most popular among them. There are also some tools like Cobertura, Arquillian etc.

A full example using JUnit, JaCoCo and Maven for code coverage is given here: https://softwarecave.org/2014/02/01/using-junit-jacoco-and-maven-for-code-coverage/

Full Code is given here: https://github.com/robertp1984/softwarecave/tree/master/jacoco