dotnet cli - specified .runsettings file not used in code coverage run

Question

I have a solution containing two dotnet core 2.1 projects (c#).

1. The first is a console application

2. The seconds is a test project with unit tests

I generate code coverage stats about project 1 when executing tests in project 2 using this command:

dotnet test C:\tempDir\SampleApp\Tests\SampleApp.Tests.csproj 
/p:CollectCoverage=true /p:CoverletOutputFormat=cobertura 
/p:CoverletOutput=C:\tempDir\Coverage\coverage 
/p:settings=CodeCoverage.runsettings --filter Category=Unit --logger trx 
--results-directory C:\tempDir\output

You can see here I specify CodeCoverage.runsettings as the settings parameter - /p:settings=CodeCoverage.runsettings. In my run settings file, I've asked that Program.cs and Startup.cs are excluded from coverage, but they are still included in the output coverage.cobertura.xml file.

Extract from output report below:

<classes>
    <class name="SampleApp.Startup" filename="SampleApp\Startup.cs" line-rate="1" branch-rate="0" complexity="2">
      <methods>
        <method name="ConfigureAppConfiguration" signature="(Microsoft.Extensions.Configuration.IConfigurationBuilder)" line-rate="1" branch-rate="0">
          <lines>
            <line number="18" hits="1" branch="False" />
            <line number="19" hits="1" branch="False" />
            <line number="20" hits="1" branch="False" />
          </lines>
        </method>
        <method name="ConfigureLogging" signature="(Microsoft.Extensions.Configuration.IConfiguration,Microsoft.Extensions.Logging.ILoggingBuilder)" line-rate="1" branch-rate="0">
          <lines>
            <line number="23" hits="1" branch="False" />
            <line number="24" hits="1" branch="False" />
            <line number="25" hits="1" branch="False" />
            <line number="26" hits="1" branch="False" />
            <line number="27" hits="1" branch="False" />
          </lines>
        </method>
      </methods>
      <lines>
        <line number="18" hits="1" branch="False" />
        <line number="19" hits="1" branch="False" />
        <line number="20" hits="1" branch="False" />
        <line number="23" hits="1" branch="False" />
        <line number="24" hits="1" branch="False" />
        <line number="25" hits="1" branch="False" />
        <line number="26" hits="1" branch="False" />
        <line number="27" hits="1" branch="False" />
      </lines>
    </class>
</classes>

I'm wondering what I've done wrong in my runsettings file? (contents of file below)

<?xml version="1.0" encoding="utf-8"?>

<RunSettings>
    <!-- Configurations for data collectors -->
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
                <Configuration>
                    <CodeCoverage>

                        <ModulePaths>
                            <Include>
                                <ModulePath>.*dll$</ModulePath>
                            </Include>
                            <Exclude>
                                <ModulePath>.*microsoft.*</ModulePath>
                                <ModulePath>.*moq.*</ModulePath>
                                <ModulePath>.*polly.*</ModulePath>
                                <ModulePath>.*fluentassertions.*</ModulePath>
                                <ModulePath>.*newtonsoft.*</ModulePath>
                                <ModulePath>.*SampleApp.Tests.*</ModulePath>
                                <ModulePath>.*\\[^\\]*DocumentManagement[^\\]*\.dll</ModulePath>
                            </Exclude>
                        </ModulePaths>

                        <Functions>
                            <Exclude>
                                <Function>.*\.Program\..*</Function>
                                <Function>.*\.Startup\..*</Function>
                                <Function>.*\.SomeOtherClass\..*</Function>
                            </Exclude>
                        </Functions>

                        <Attributes>
                            <Exclude>
                                <Attribute>^System\.Diagnostics\.DebuggerHiddenAttribute$</Attribute>
                                <Attribute>^System\.Diagnostics\.DebuggerNonUserCodeAttribute$</Attribute>
                                <Attribute>^System\.Runtime\.CompilerServices.CompilerGeneratedAttribute$</Attribute>
                                <Attribute>^System\.CodeDom\.Compiler.GeneratedCodeAttribute$</Attribute>
                                <Attribute>^System\.Diagnostics\.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>
                            </Exclude>
                        </Attributes>

                        <!-- We recommend you do not change the following values: -->
                        <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
                        <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
                        <CollectFromChildProcesses>True</CollectFromChildProcesses>
                        <CollectAspDotNet>False</CollectAspDotNet>

                    </CodeCoverage>
                </Configuration>
            </DataCollector>

        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

Not sure why this section is still here in this output report, when I specified it being skipped in the runsettings file.

NOTE: I'm trying to avoid littering my code with the [ExcludeFromCodeCoverage] attribute and I don't want to have to start adding /p:ExcludeByFile=Program.cs or /p:ExcludeByFile=Startup.cs to my test command in builds, hence using the runsettings file.

Answer 1

You cannot exclude classes using runsettings file by just providing the class name.

The Function element from run setting matches the full name of a a function / method like

YourNamespace.YourClass.Method(parameters);

There are only below settings which are possible from the documentation :

Other ways to include or exclude elements ModulePath - matches assemblies specified by assembly file path.

CompanyName - matches assemblies by the Company attribute.

PublicKeyToken - matches signed assemblies by the public key token.

Source - matches elements by the path name of the source file in which they are defined.

Attribute - matches elements to which a particular attribute is attached. Specify the full name of the attribute, and include "Attribute" at the end of the name.

Function - matches procedures, functions, or methods by fully qualified name. To match a function name, the regular expression must match the fully qualified name of the function, including namespace, class name, method name, and parameter list.

What options you have:

Option 1: Using Starts with OR Using method name

        <Functions>
          <Exclude>

            <!-- Exclude all methods in SampleApp.Program : -->
            <Function>^SampleApp\.Program\..*</Function>

            <!-- Exclude all methods named Main: -->
            <Function>.*\.Main\(.*</Function>
          </Exclude>
        </Functions>

In first Function, please note that your namespace with class name is specified and it is starting with ^ character.

In second function element, please note that it is checking method name by checking if a string ends with opening parenthesis '('.

Option 2: You can use attributes on classes and exclude them from them runsettings file.

This is similar to ExcludeFromCodeCoverate attribute.

Please note the complete runsettings file at the end of this documentation page.