Printing number of JUnit failures in Ant

Question

<target name="test" depends="compile-test">
    <junit failureProperty="test.failure">
      <classpath refid="classpath.test" />

      <formatter type="brief" usefile="false" />
      <batchtest>
        <fileset dir="${tst-dir}" includes="**/Test*.class" />
      </batchtest>
    </junit>

    <fail message="test failed" if="test.failure" />
  </target>

I want to print how many test cases are:

1. failed
2. error
3. passed

by making changes only in the build.xml file. How can I do that?

Answer 1

You can use the junitreport task to gather your test results.

If you need to print the summary metrics in your build file, you could use a filter chain to extract the information from the generated report.

There may (must?) be a simpler way to do this, but I didn't see it.

<target>
    <junit failureProperty="test.failure">
        <classpath refid="classpath.test" />

        <!-- use XML formatter and let it output to file -->
        <formatter type="xml" />

        <!-- specify output dir for test result files -->
        <batchtest todir="tmp/results">
            <fileset dir="${tst-dir}" includes="**/Test*.class" />
        </batchtest>
    </junit>

    <!-- generate report with junitreport -->
    <junitreport todir="tmp">
        <fileset dir="tmp/results" />
        <report todir="tmp/report" />
    </junitreport>

    <!-- concat the report through a filter chain to extract what you want -->
    <concat>
        <fileset file="tmp/report/overview-summary.html" />
        <filterchain>
            <linecontainsregexp>
                <regexp pattern='title="Display all tests"' />
            </linecontainsregexp>
            <tokenfilter>
                <replaceregex pattern='&lt;td&gt;&lt;a href="all-tests.html" title="Display all tests"&gt;(\d+)&lt;/a&gt;&lt;/td&gt;&lt;td&gt;&lt;a href="alltests-fails.html" title="Display all failures"&gt;(\d+)&lt;/a&gt;&lt;/td&gt;&lt;td&gt;&lt;a href="alltests-errors.html" title="Display all errors"&gt;(\d+).*$' replace="Run: \1, Failed: \2, Errors: \3" />
            </tokenfilter>
        </filterchain>
    </concat>

    <fail message="test failed" if="test.failure" />
</target>

The output will be something like:

Buildfile: C:\\test\unit_test.xml
test:
    [junit] Test MyUnitTest FAILED
    [junit] Test MyUnitTest2 FAILED
[junitreport] Processing C:\\test\tmp\TESTS-TestSuites.xml to C:\DOCUME~1\xxx\LOCALS~1\Temp\1\null1075123857
[junitreport] Loading stylesheet jar:file:/C:/eclipse/eclipse-jee-ganymede-SR2-win32/eclipse/plugins/org.apache.ant_1.7.0.v200803061910/lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl
[junitreport] Transform time: 906ms
[junitreport] Deleting: C:\DOCUME~1\xxx\LOCALS~1\Temp\1\null1075123857
   [concat] Run: 8, Failed: 4, Errors: 1

BUILD FAILED
C:\test\unit_test.xml:32: test failed

Total time: 1 second

If you are running a large number of tests, you will now have the overhead of report generation extraction.