Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

view code coverage report on azure devops portal

I am running the NUnit tests (project in .Net Framework 4.5), as part of azure devops build pipeline.

- task: VSTest@2
  inputs:
    testAssemblyVer2: 'tests/**/*.Tests.dll'
    pathtoCustomTestAdapters: '$(Build.SourcesDirectory)/packages'
    codeCoverageEnabled: true
  displayName: 'NUnit Testing'

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: JaCoCo
    summaryFileLocation: '$(Common.TestResultsDirectory)/**/*.xml'
  displayName: 'Publish Code Coverage'
  //        summaryFileLocation: '$(Common.TestResultsDirectory)/**/*.coverage'

But I am not able to see the coverage report, all I see the download link for coverage results...

code coverage download link

How can I convert the .coverage report to JaCoCo format? OR generate the report directly in JaCoCo format?

I have seen some solution for .Net Core (link), but none for .Net framework

like image 535
harishr Avatar asked Feb 11 '19 09:02

harishr


People also ask

How do I get code coverage report in Azure DevOps?

The code coverage summary can be viewed on the Summary tab on the pipeline run summary. The results can be viewed and downloaded on the Code coverage tab.

How do I get code coverage results?

Code coverage results are displayed in the Coverage tool window, in the Project tool window, and in the editor after you run at least one configuration with coverage. Results of the code coverage analysis are saved to the coverage folder in the IDE system directory.

How do I see Azure DevOps reports?

In-context reports: Pipelines and Test Several in-context reports are provided for Azure Pipelines. These reports derive from Analytics data. Open a pipeline (or release summary for Test failure) to view the reports and select the Analytics tab. Select View full report on a summary card for a detailed report.


2 Answers

Update:

As per the release to Azure Devops for Sprint 150

When publishing code coverage reports, you no longer need to specify HTML files.

Therefore, the script in my illustration no longer needs to use the report generator tool directly to create the html report, and when publishing the coverage results, the directory containing those html reports doesn't need to be specified.

Edit:


The trick I've found for getting the coverage results from a .Net Framework project to show up on the Code Coverage tab is in the same line of thought to your linked article.

  1. Don't run tests with the VS Test Task in Azure
  2. Install the Report Generator and Coverlet tools directly
  3. Use dotnet-vstest command for running tests through Coverlet
  4. Publish reports generated with Report Generator and Cobertura format coverage results


Don't use the VS Test Task

Running this task will allow you to collect coverage with a simple checkbox, but you then surrender your opportunity to provide the content for the Code Coverage Tab

no VsTest task



Install tools directly

Use a Powershell task (or similar) to install the Coverlet and Report Generator tools directly. This allows you to use them on projects that are not .Net Core.

"install tools:"
&dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.12
&dotnet tool install coverlet.console --tool-path . --version 1.4.1



Use dotnet vstest through coverlet

It's my understanding that dotnet test doesn't play nice with .Net Framework projects/assemblies. However, we can still use the dotnet command, which we know will be on the agent machine, but we need to use it as a mechanism to get to the vstest.console.exe.

The Coverlet tool, as mentioned in the article you linked, will output coverage results in Cobertura format if you tell it to do so.

&$coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"


Publish results

use publish code coverage task



Complete script sample

note: this script is pretty rough, so use it as a thought exercise for your individual situation.

"install tools:"
&dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.12
&dotnet tool install coverlet.console --tool-path . --version 1.4.1

"`nmake reports dir:"
mkdir .\reports

"`nrun tests:"
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*UnitTestProject2.dll" }
Write-Host "`$unitTestFile value: $unitTestFile"

$coverlet = "$pwd\coverlet.exe"

"calling $coverlet for $($unitTestFile.FullName)"
&$coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"

"`ngenerate report(s)"
gci -Recurse | 
    ?{ $_.Name -eq "coverage.cobertura.xml" } | 
    %{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reporttypes:HTMLInline;HTMLChart" }

If you're struggling to figure out the escaping of quotes and such with the Coverlet command, YOU ARE NOT ALONE. I used the echoargs commandlet from PSCX more times than I care to admit so I could see what was actually getting provided to the .exe calls I was making.



The Results!!

...because that's really what matters

enter image description here



enter image description here




Original Answer:


Because of the way the linked article you mentioned is installing and using the report generator global tool I would think you can still follow those guidelines for creating the HTML inline and chart report types.

I'm not sure what is meant or how it works when the article says,

The point is the reporttypes: Use HTMLInLine for enabling the output on the Azure DevOps page. Azure DevOps Coverage page show index.html on the web.

I'm understanding that you can use the tool to create the HTML report from the .xml coverage results, and then publish the coverage results and report together with the Publish Code Coverage task.

So it seems all you need is to have an .xml format of the .coverage tool.

I didn't get it working in straight powershell, but you could follow the instructions from the Report Generator documentation to write a C# utility to access the Coverage.Analysis library.

like image 73
Josh Gust Avatar answered Oct 22 '22 09:10

Josh Gust


For anyone looking for code coverage in Azure Devops (using classic editor, without Yaml), in current .NET (core) 5, with xUnit tests:

  1. In your xUnit test project, add following (it generally comes by default in .NET 5, xUnit template now):

    <PackageReference Include="coverlet.collector" Version="3.0.3" />

    Keep checking for new version.

  2. Head to Azure devops, create pipeline using classic editor. Do the restore, build steps. (Or you can choose dotnet core template as below): select dotnet core pipeline template

  3. In the test command of dotnet core task, add argument - --collect:"XPlat Code Coverage". Remember "XPlat Code Coverage" is friendly name and case sensitive. Your test command would look like: enter image description here Check this checkbox if you want to publish your test results: Publish test results and code coverage, but it won't publish code coverage. The functionality is not yet working (at least not in non-windows).

  4. Next add - Publish code coverage results task. Choose "Code coverage tool" as "Cobertura" and in "Summary file" field, add $(Agent.TempDirectory)/**/coverage.cobertura.xml. Looks like this: dotnet test task config in pipeline

  5. Save and Queue (in any agent, I use Ubuntu) and see the result once pipeline run completes: pipeline run result

  6. Coverage report tab: code coverage summary

  7. HTML Coverage reports and coverage cobertura xml are published as artifacts: coverage reports

like image 21
Meer Avatar answered Oct 22 '22 09:10

Meer