Fail a build if code coverage is below a threshold in TFS2012

Question

I'm trying to fail builds in TFS Service (Hosted TFS2012) when Code Coverage is below a threshold.

I've been messing around with the solution at http://scrumdod.blogspot.co.uk/2011/04/fail-build-if-code-coverage-is-low.html

However, I'm using TFS2012 and a great many things appear to have changed. In particular, the configuration of the test run is completely different and there does not appear to be any way to get or set the location and name of the .coverage file in either the build process template or the .runsettings file.

How would I go about finding (or setting) the location of the .coverage file in TFS2012 or TFSService?

Alterntely, is there another way that I can fail the build if code coverage is below a threshold?

Answer 1

This will require a few steps:

1. Create a custom build activity
2. Add the activity to the build controller
3. Invoke that custom build activity within a new build process
4. Use the new build process

1. Create a custom build activity.

Make a new project in VS2012 (I called mine CodeCoverageLibrary. Reference the following assemblies:

• Microsoft.TeamFoundation.Build.Client
• Microsoft.TeamFoundation.Client
• Microsoft.TeamFoundation.TestManagement.Client
• Microsoft.TeamFoundation.WorkItemTracking.Client
• System
• System.Activities
• System.Core
• System.Xaml

You can use the following code:

using System;
using System.Activities;
using System.Collections.Generic;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;

namespace CodeCoverageLibrary
{
    [BuildActivity(HostEnvironmentOption.All)]
    public sealed class GetCodeCoverage : CodeActivity<double>
    {
        public InArgument<IBuildDetail> BuildDetail { get; set; }

        protected override double Execute(CodeActivityContext context)
        {
            var buildDetail = BuildDetail.Get(context);
            var buildCoverages = GetBuildCoverages(buildDetail.BuildServer.TeamProjectCollection.Uri,
                                                   buildDetail.TeamProject, buildDetail.Uri);
            return GetTotalCoverage(buildCoverages);
        }

        private static IEnumerable<IBuildCoverage> GetBuildCoverages(Uri uri, string projectName, Uri buildUri)
        {
            return TfsTeamProjectCollectionFactory.GetTeamProjectCollection(uri)
                                                  .GetService<ITestManagementService>()
                                                  .GetTeamProject(projectName)
                                                  .CoverageAnalysisManager.QueryBuildCoverage(buildUri.ToString(),
                                                                                              CoverageQueryFlags.Modules);
        }

        private static double GetTotalCoverage(IEnumerable<IBuildCoverage> buildCoverages)
        {
            int totalCoveredBlocks = 0, totalUncoveredBlocks = 0;
            foreach (var buildCoverage in buildCoverages)
            {
                foreach (var module in buildCoverage.Modules)
                {
                    totalCoveredBlocks += module.Statistics.BlocksCovered;
                    totalUncoveredBlocks += module.Statistics.BlocksNotCovered;
                }
            }

            return (totalCoveredBlocks == 0 && totalUncoveredBlocks == 0)
                                      ? 0.0
                                      : ((double) totalCoveredBlocks) /
                                        ((double) (totalCoveredBlocks + totalUncoveredBlocks));
        }
    }
}

Compile the project and add to to a version controlled path on TFS.

2. Add the activity to the build controller

In Team Explorer, go to Builds > Actions > Manage Build Controllers .... Then click on Properties... for the build controller. Under Version control path to custom assemblies, put the path you used above.

3. Invoke that custom build activity within the build process

Copy BuildProcessTemplates\DefaultTemplate.11.1.xaml to a new file.

Update the beginning of the new XAML file to include the following:

<Activity ...
          xmlns:ccl="clr-namespace:CodeCoverageLibrary;assembly=CodeCoverageLibrary"
          ...
          >
  <x:Members>
    ...
    <x:Property Name="CodeCoverageTolerance" Type="InArgument(x:Double)" />
  </x:Members>
  ...
  <this:Process.Metadata>
    <mtbw:ProcessParameterMetadataCollection>
      ...
      <mtbw:ProcessParameterMetadata BrowsableWhen="EditingDefinition" Category="#900 Misc" DisplayName="CodeCoverageTolerance" Description="If the overall code coverage drops below this threshold, then the build will fail. This is a number between 0 and 1." ParameterName="CodeCoverageTolerance" />
    </mtbw:ProcessParameterMetadataCollection>
  </this:Process.Metadata>

Update the end of the XAML file to include the following:

      <Sequence DisplayName="Code Coverage Check" mtbwt:BuildTrackingParticipant.Importance="None">
        <Sequence.Variables>
          <Variable x:TypeArguments="x:Double" Name="CodeCovered" />
        </Sequence.Variables>
        <ccl:GetCodeCoverage DisplayName="Getting Code Coverage" BuildDetail="[BuildDetail]" Result="[CodeCovered]" />
        <If Condition="[CodeCovered &lt; CodeCoverageTolerance]">
          <If.Then>
            <Sequence DisplayName="Comparing Code Coverage Against Tolerance">
              <mtbwa:SetBuildProperties DisplayName="Set TestStatus to Failed" mtbwt:BuildTrackingParticipant.Importance="Low" PropertiesToSet="TestStatus" TestStatus="[Microsoft.TeamFoundation.Build.Client.BuildPhaseStatus.Failed]" />
              <mtbwa:WriteBuildError Message="[&quot;Code Coverage of &quot; + CodeCovered.ToString(&quot;P&quot;) + &quot; is less than required &quot; + CodeCoverageTolerance.ToString(&quot;P&quot;)]" />
            </Sequence>
          </If.Then>
        </If>
      </Sequence>
    </mtbwa:AgentScope>
    <mtbwa:InvokeForReason DisplayName="Check In Gated Changes for CheckInShelveset Builds" Reason="CheckInShelveset">
      <mtbwa:CheckInGatedChanges DisplayName="Check In Gated Changes" />
    </mtbwa:InvokeForReason>
  </Sequence>
</Activity>

Check this into TFS.

4. Use the new build process

In Team Explorer, go to Builds and right-click on your build definition. Go to Edit Build Definition... > Process. Expand Build process template and click New.... Click Select an existing XAML file and put the path to the new XAML file. Click OK. You should now see CodeCoverageTolerance under 4. Misc. You can put in a number between 0 and 1 to indicate your desired percentage.