How to define multiple matching patterns in AzureDevOps Pipeline VSTest@2?

Question

Currently I'm trying to set up a new pipeline for our solution and can't get the Visual Studio Test to find the correct set of tests within my solution. Either it picks a DLL that doesn't contain any tests (which leads to a fail of the task) or if I specify the testAssemblyVer2 property it produces a warning that it couldn't find any assembly to test.

The base task configuration we are working with:

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'
    searchFolder: '$(System.DefaultWorkingDirectory)'
    runInParallel: true
    codeCoverageEnabled: true
    diagnosticsEnabled: true

If we run this, we can see in the output the following configuration (part):

 ...
 Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
 ...
 ======================================================
 [command]"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" @d:\a\_temp\66884a11-77b3-11e9-b7cb-25533524cce5.txt
 Microsoft (R) Test Execution Command Line Tool Version 16.0.1
 Copyright (c) Microsoft Corporation.  All rights reserved.

 "d:\a\1\s\Tests\Api\FirstController.Tests\bin\Release\netcoreapp2.1\FirstController.Tests.dll"
 "d:\a\1\s\Tests\Api\SecondController.Tests\bin\Release\netcoreapp2.1\SecondController.Tests.dll"
 "d:\a\1\s\Tests\CreateTranslateStringsFromDeviceConfigurationSettings\bin\Release\netcoreapp2.1\CreateTranslateStringsFromDeviceConfigurationSettings.dll"
 "d:\a\1\s\Tests\Api\FourthController.Tests\bin\Release\netcoreapp2.1\FourthController.Tests.dll"
 "d:\a\1\s\Tests\Api\FifthController.Tests\bin\Release\netcoreapp2.1\FifthController.Tests.dll"
 /Settings:"d:\a\_temp\69a604d0-77b3-11e9-b7cb-25533524cce5.runsettings"
 /EnableCodeCoverage
 /logger:"trx"
 /TestAdapterPath:"d:\a\1\s"
 Starting test execution, please wait...

As you can see, there is one assembly CreateTranslateStringsFromDeviceConfigurationSettings that doesn't contain any tests but is picked as a candidate for tests. I took the exact original name from my concrete solution just to show that it obviously doesn't match the pattern but is picked. Now we try to avoid this problem by defining our own match pattern.

If we create the task through the assistant, it will add the following value by default:

    testAssemblyVer2: '**\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\**'

If we run this we get the following output:

...
 Test assemblies : **\*test*.dll !**\*TestAdapter.dll !**\obj\**
...
 ##[warning]No test assemblies found matching the pattern: **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**.

In the output you can see, that the list of test assemblies isn't comma separated, which gives some indication that the value is not understood correctly and therefore maybe leads to the empty list.

So we try to simply copy and paste the comma values from the first running output which produces the following configuration and (failed) output:

    testAssemblyVer2: '**\*test*.dll,!**\*TestAdapter.dll,!**\obj\**'

Output:

 ...
 Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
 ...
 ##[warning]No test assemblies found matching the pattern: **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**.

The output matches the first now, but it still doesn't work. So using commas doesn't seem to be the way to go.

So in a fourth case I took the value from the documentation which is

testAssemblyVer2: '**\*test*.dll!**\*TestAdapter.dll!**\obj\**'

But it also failed with the similar error message:

...
Test assemblies : **\*test*.dll!**\*TestAdapter.dll!**\obj\**
...
##[warning]No test assemblies found matching the pattern: **\*test*.dll!**\*TestAdapter.dll!**\obj\**.

So how to define multiple patterns the correct way?

Answer 1

Try this:

- task: VSTest@2
  inputs:
    testAssemblyVer2: |
     **\*test.dll
     !**\*TestAdapter.dll
     !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'