Is it possible to use MSTest V2 with Visual Studio 2017 Test Controller and Test Agent for remotely executing unit tests

Question

Goal

I want to be able to run my unit tests remotely on another machine since they interact with the UI of another application. For clarity these tests are not Coded UI Tests, they are tests methods that use FlaUI to interact with the desktop.

Problem

I can't get the Visual Studio Test Controller and Test Agent to work with MSTest V2. When I set the .runsettings file to use the .testsettings file and to ForcedLegacyMode like the documentation says here I get the following warnings and no tests are loaded into the test explorer.

[11/22/2017 9:54:12 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:13 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:13 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:14 AM Warning] Warning : A testsettings file or a runsettings with a ForcedLegacyMode set to true is not supported with the MSTest V2 Adapter.

I am hoping I am just missing some setting I can put into my .runsettings file that will allow me to specify the url for my Test Controller.

Settings

Here are my .runsettings and .testsettings files for reference. These settings successfully connect to the machine but when I build my test runner no longer finds and tests to run.

.runSettings

  <?xml version="1.0" encoding="utf-8"?>
  <RunSettings>
    <!-- Configurations that affect the Test Framework -->
    <RunConfiguration>
      <MaxCpuCount>1</MaxCpuCount>
      <!-- Path relative to solution directory -->
      <ResultsDirectory>.\TestResults</ResultsDirectory>

      <!-- [x86] | x64    
        - You can also change it from menu Test, Test Settings, Default Processor Architecture -->
      <TargetPlatform>x86</TargetPlatform>

      <!-- Framework35 | [Framework40] | Framework45 -->
      <TargetFrameworkVersion>Framework40</TargetFrameworkVersion>
    </RunConfiguration>

    <!-- Configurations for data collectors -->
    <DataCollectionRunSettings>
      <DataCollectors>
        <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
          <Configuration>
            <CodeCoverage>
              <ModulePaths>
                <Exclude>
                  <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
                </Exclude>
              </ModulePaths>

              <!-- We recommend you do not change the following values: -->
              <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
              <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
              <CollectFromChildProcesses>True</CollectFromChildProcesses>
              <CollectAspDotNet>False</CollectAspDotNet>

            </CodeCoverage>
          </Configuration>
        </DataCollector>

      </DataCollectors>
    </DataCollectionRunSettings>

    <!-- Parameters used by tests at runtime -->
    <TestRunParameters>
    </TestRunParameters>

    <!-- Adapter Specific sections -->

    <!-- MSTest adapter -->
    <MSTest>
      <MapInconclusiveToFailed>True</MapInconclusiveToFailed>
      <CaptureTraceOutput>false</CaptureTraceOutput>
      <DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
      <DeploymentEnabled>False</DeploymentEnabled>
    </MSTest>

  </RunSettings>

.testsettings

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="rmoqa01" id="076be28c-d18b-46bf-ad20-4d43ec821ea4" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>These are default test settings for a local test run.</Description>
  <RemoteController name="10.2.0.101" />
  <Execution location="Remote">
    <Hosts skipUnhostableTests="false">
      <VSSDKTestHostRunConfig name="VS IDE" HiveKind="DevEnv" HiveName="15.0_c9b36733" xmlns="http://microsoft.com/schemas/VisualStudio/SDK/Tools/IdeHostAdapter/2006/06" />
    </Hosts>
    <TestTypeSpecific>
      <UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
        <AssemblyResolution>
          <TestDirectory useLoadContext="true" />
        </AssemblyResolution>
      </UnitTestRunConfig>
    </TestTypeSpecific>
    <AgentRule name="AllAgentsDefaultRole">
    </AgentRule>
  </Execution>
  <Properties />
</TestSettings>

Answer 1

I opened a issue on the mstest github page and after looking at the source code generating my warning I am seeing it looks like there is probably no work around to this. Here is the source code I was looking at in the MSTestDiscover.cs.

    // Scenarios that include testsettings or forcing a run via the legacy adapter are currently not supported in MSTestAdapter.
    if (MSTestSettings.IsLegacyScenario(logger))
    {
        return;
    }

EDIT: 10/28/2018

The issue I linked to above was updated with the following response.

There are no plans to have remote execution capability through MSTVest2. You can use the test task in VSTS which supports distributed execution on multiple agents. https://docs.microsoft.com/en-us/vsts/pipelines/test/set-up-continuous-test-environments-builds?view=vsts