An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem

Question

I'm facing to an issue when I'm running my unit tests on a VSTS build. When I'm launching all my tests on Visual Studio locally, everything works fine. No error at all. Perfect. But when I'm launching my tests via a tests task on a VSTS build, I have this error:

An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=***********' or one of its dependencies. The system cannot find the file specified.

My test project is set like this:

• "xunit.runner.visualstudio" on version 2.4.0 with a "targetFramework" set to "net462" (set inside the packages.config)

• .NET Framework 4.6.2

On the build server I have:

• Visual Studio 2017, v15.7.5

• Visual Studio 2015, v14.0

I'm running out of clues... I'm surely missing something... Thanks for your help

PS:

• All tests passed correctly on the VSTS build
• On the settings of my build task, when I set "Test platform version" to "Visual Studio 2015", it works. But when I'm choosing "Visual Studio 2017" or "Latest" I have the same error...

EDIT

I have Nuget packages "FluentAssertions" v5.4.1 and "System.ValueTuple" v4.5.0. I already tried to downgrade one of them and both. It failed. But I saw that "FluentAssertions" has a dependency to "System.ValueTuple"

EDIT 2

As @PatrickLu-MSFT said, passing the vs test task version to 1 solved the issue. But when I'm looking closely, the Test platform versions available are 2015 and 2013:

When I'm choosing VS test task version 2, I have those choices:

I would like to stay on vs test task version 2 (in case of future updates, I don't want to be stuck...). What I saw is when I'm on Test platform version set to "Latest" or "Visual Studio 2017", it does not work. When I'm choosing "Visual Studio 2015", it working... So what I point is:

• Having a vs test task version set to version 1 make it work (maybe because the latest test platform version is "Visual Studio 2015"...?)
• Having a vs test task version set to version 2 with Test platform version set to "Visual Studio 2015" make it work

So:

• A VS test task version set to version 1 seems to work the same way as a VS test task version set to version 2 with a "Test platform version" set to "Visual Studio 2015".
• How can I make it work with a VS test task version set to version 2 and a "Test platform version" set to "Latest"?

EDIT 3

After connected remotely to the server of my build agent and run tests from VS2017, I compare the output of Visual Studio 2017 of the host server to the output of my local Visual Studio 2017:

• Local Visual Studio 2017:

• Remote Visual Studio 2017:

Are there some issues with "xUnit.net VSTest Adapter v2.4.0 (32-bit Universal Windows)" on the host?

Answer 1

I hit this same problem and solved it by changing the "Path to custom test adapters" to point specifically to the correct test adapter by setting it to "Source\packages\xunit.runner.visualstudio.2.4.0\build_common". (You should modify the path to account for where the packages folder is in your repository.)

This apparently happens because by default VSTS tries to be helpful and scans the entire working directory structure for test adapters and loads all of them, but the xUnit Visual Studio test adapter Nuget package installs test adaptors for .Net Core and UWP as well, so VSTS happily loads them and they blow up.