How to fetch and parse all the generated coverage.cobertura files in CI pipelines?

Question

Given a .Net 5 solution with multiple xUnit test projects I can run dotnet test from the root of the solution and it will run all the tests.

I would like to generate reports so based on https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-code-coverage?tabs=windows#integrate-with-net-test I run dotnet test --collect:"XPlat Code Coverage" which generates a coverage.cobertura file per test project.

Based on https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-code-coverage?tabs=windows#generate-reports I know that I can install the dotnet-reportgenerator-globaltool tool and get a visual HTML report based on each coverage.cobertura file.

But I want to add a CI pipeline where I want to make the pipeline fail when the line coverage is below x %.

Given the following sample Gitlab CI configuration

image: mcr.microsoft.com/dotnet/sdk:5.0

stages:
  - tests

tests:
  stage: tests
  script:
    - dotnet test --collect:"XPlat Code Coverage"

how can I gather all the generated coverage.cobertura.xml files, read the line coverage and let the pipeline fail if the value is below e.g. 80%?

Example:

tests:
  stage: tests
  script:
    - dotnet test --collect:"XPlat Code Coverage"
    # for each coverage.cobertura file found in the test projects
    # parse the file
    # read the line coverage
    # fail if the value is less than 80 percent

It would be nice if I don't have to reinvent the wheel if tools like xUnit already provide such functionality!

---

Edit:

I know that I could also use the allow_failure keyword to leave this stage in a warning state. This would be fine for me, I just want to know how to read the required information from the generated reports and validate them to decide if that stage should pass, fail or be unstable.

Answer 1

This might sound like a really novice approach, but here is something that worked for me. I am using Azure devops and I had similar multiple projects, resulting in multiple coverage.cobertura file per test project

First I used Report Generator task to merge all coverage reports into one, and store it in working directory. Below is the yaml. I'm generating both, HtmlFormat as well as Cobertura report

- task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
  displayName: ReportGenerator
  inputs:
    reports: '$(Build.SourcesDirectory)/**/*cobertura.xml'
    targetdir: '$(System.DefaultWorkingDirectory)/CoverageResults'
    reporttypes: 'HtmlInline_AzurePipelines;Cobertura'

Resulting cobertura report looks like this:

After this, I tried reading this merged xml report using powershell task

- powershell: |
   [XML]$coverage = Get-Content $(System.DefaultWorkingDirectory)/CoverageResults/Cobertura.xml
   if($coverage.coverage.'line-rate' -ge .50){Write-Host "The value is greater than 50."}else{throw}
  displayName: 'PowerShell Script'

In this task, I'm trying to read the coverage file, and then accessing the line-rate attribute. If line rate of merged report is not greater than or equal to 50, I'm throwing error. With ErrorActionPreference set to Stop for my powershell task, the pipeline stops if line coverage is not as per expectation. You can also include other conditions like branch rate for better accuracy .

There is a lot to improve in this I guess, but this was just a quick workaround that I could come up with. Please feel free to suggest improvements