Using MBUnit in TeamCity

Question

I'm compiling a NAnt project on linux with TeamCity Continuous Integration server. I have been able to generate a test report by running NAnt on mono thru a Command Line Runner but don't have the options of using the report like a NAnt Runner. I'm also using MBUnit for the testing framework.

How can I merge in the test report and display "Tests failed: 1 (1 new), passed: 3049" for the build?

Update: take a look at MBUnitTask its a NAnt task that uses sends messages that TeamCity expects from NUnit so it lets you use all of TeamCity's features for tests.

MBUnitTask

Update: Galio has better support so you just have to reference the Galio MBUnit 3.5 dlls instead of the MBUnit 3.5 dlls and switch to the galio runner to make it work.

Answer 1

Gallio now has an extension to output TeamCity service messages. Just use the included Gallio.NAntTasks.dll and enable the TeamCity extension. (this won't be necessary in the next release)

Answer 2

TeamCity watches the command line output from the build. You can let it know how your tests are going by inserting certain markers into that output See http://www.jetbrains.net/confluence/display/TCD3/Build+Script+Interaction+with+TeamCity. For example

##teamcity[testSuiteStarted name='Test1']

will let TeamCity know that a set of tests started. With MbUnit you can't output these markers while the tests are running, but you can transform the XML file that it outputs. Here is the XSL that I am using:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text"/>
    <xsl:template match="/">

        <xsl:apply-templates/>

    </xsl:template>

    <xsl:template match="assemblies/assembly">
##teamcity[testSuiteStarted name='<xsl:value-of select="@name" />']

        <xsl:apply-templates select="//run" />

##teamcity[testSuiteFinished name='<xsl:value-of select="@name" />']
    </xsl:template>

    <xsl:template match="run">

        <xsl:choose>
            <xsl:when test="@result='ignore' or @result='skip'">
        ##teamcity[testIgnored name='<xsl:value-of select="@name" />' message='Test Ignored']
            </xsl:when>
            <xsl:otherwise>
        ##teamcity[testStarted name='<xsl:value-of select="@name" />']
            </xsl:otherwise>
        </xsl:choose>


        <xsl:if test="@result='failure'">
            ##teamcity[testFailed name='<xsl:value-of select="@name" />' message='<xsl:value-of select="child::node()/message"/>' details='<xsl:value-of select="normalize-space(child::node()/stack-trace)"/>']
        </xsl:if>


        <xsl:if test="@result!='ignore' and @result!='skip'">
        ##teamcity[testFinished name='<xsl:value-of select="@name" />']
        </xsl:if>

    </xsl:template>

</xsl:stylesheet>

Answer 3

Here's what I came up with

How can I merge in the test report?

First you'll need to get mbunit to generate both an XML and HTML report. The Command line arguments look like this

/rt:Xml /rt:Html /rnf:mbunit /rf:..\reports

this will generate the reports into a dir called reports and the file will be called mbunit.xml and mbunit.html

next we want to add these files as artifacts on the build

build\reports\* => Reports

the last step is to tell teamcity to add it as a tab for the build

find the .BuildServer\config\main-config.xml and add this line (on windows this is in c:\Documents and Settings\, on linux it was in the /root dir)

<report-tab title="Tests" basePath="Reports" startPage="mbunit.html" />

How can I display "Tests failed: 1 (1 new), passed: 3049" for the build?

TeamCity looks for a file called teamcity-info.xml where you can stick messages in to be displayed. The Actual test count is actually just plain text. I think you can just add the file as an artifact but I've also got it in the root dir of the build.

in NAnt you'll want to use this command to do an XSLT on the MBUnit XML Report

<style style="includes\teamcity-info.xsl" in="reports\mbunit.xml" out="..\teamcity-info.xml" />

the actual xsl looks like this. (Note: that the { and } are reserved in xsl so we have to use params)

<?xml version="1.0" encoding="ISO-8859-1"?>

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:param name="cbl" select="'{'"/>
<xsl:param name="cbr" select="'}'"/>
<xsl:template match="/">
<xsl:for-each select="report-result/counter">

<build number="1.0.{concat($cbl,'build.number',$cbr)}">
    <xsl:if test="@failure-count &gt; 0">
        <statusInfo status="FAILURE">    
            <text action="append"> Tests failed: <xsl:value-of select="@failure-count"/>, passed: <xsl:value-of select="@success-count"/></text>
        </statusInfo>
    </xsl:if>
    <xsl:if test="@failure-count = 0">
        <statusInfo status="SUCCESS">
            <text action="append"> Tests passed: <xsl:value-of select="@success-count"/></text>
        </statusInfo>
    </xsl:if>

</build>
</xsl:for-each>

</xsl:template>
</xsl:stylesheet>

This will give you a file that looks like this

<build number="1.0.{build.number}">
   <statusInfo status="FAILURE">
      <text action="append">Tests failed: 16, passed: 88</text>
   </statusInfo>
</build>