VSTS Build fails running NUnit tests

Question

When building a visual studio solution using visual studio team services it runs the unit tests, written in nunit 3 using the nunit 3 NUnit3TestAdapter it starts running the tests.

The error it gives is: Error: Illegal characters in path.

The complete output as follows

2016-02-11T17:02:40.8514411Z Executing the powershell script: C:\BuildAgent\tasks\VSTest\1.0.29\VSTest.ps1
2016-02-11T17:02:41.1483135Z ##[debug]Calling Invoke-VSTest for all test assemblies
2016-02-11T17:02:41.2108131Z Working folder: C:\BuildAgent\_work\5
2016-02-11T17:02:41.2108131Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "C:\BuildAgent\_work\5\s\xxxxxxxxxxTests\bin\Release\xxxxxxxxxxTests.dll"  /logger:trx /TestAdapterPath:"C:\BuildAgent\_work\5\s"
2016-02-11T17:02:41.5240692Z Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
2016-02-11T17:02:41.5240692Z Copyright (c) Microsoft Corporation.  All rights reserved.
2016-02-11T17:02:42.1326930Z Starting test execution, please wait...
2016-02-11T17:02:46.4921857Z Information: NUnit Adapter 3.0.8.0 executing tests is started
2016-02-11T17:02:46.5077303Z Information: Running all tests in C:\BuildAgent\_work\5\s\xxxxxxxxxxTests\bin\Release\xxxxxxxxxxTests.dll
2016-02-11T17:02:46.6327336Z Information: NUnit Adapter 3.0.8.0 executing tests is finished
2016-02-11T17:02:46.6327336Z ##[error]Error: Exception System.ArgumentException, Exception thrown executing tests
2016-02-11T17:02:46.6327336Z ##[error]
2016-02-11T17:02:46.6327336Z ##[error]Error: Illegal characters in path.
2016-02-11T17:02:46.6327336Z ##[error]
2016-02-11T17:02:46.7108474Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
2016-02-11T17:02:46.8358887Z ##[error]VSTest Test Run failed with exit code: 1
2016-02-11T17:02:46.8514965Z ##[warning]No results found to publish.

We have an on premise build agent but I don't think this should make an difference.

Any advice you can give it appreciated.

Answer 1

FWIW, I switched on verbose logging by following the Tips and Tricks and got this:

2016-02-23T14:45:49.5620657Z Information: Running all tests in C:\TFS2015\_work\3\s\BlaBlaTests\bin\Debug\BlaBlaTests.dll
2016-02-23T14:45:49.6088663Z ##[error]Error: Exception thrown executing tests
2016-02-23T14:45:49.6088663Z ##[error]
2016-02-23T14:45:49.6088663Z ##[error]Error: System.ArgumentException: Illegal characters in path.
2016-02-23T14:45:49.6088663Z ##[error]   at System.IO.Path.CheckInvalidPathChars(String path)
2016-02-23T14:45:49.6088663Z ##[error]   at System.IO.Path.Combine(String path1, String path2)
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.Engine.Services.ExtensionService.ProcessAddinsFile(DirectoryInfo baseDir, String fileName)
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.Engine.Services.ExtensionService.FindExtensionsInDirectory(DirectoryInfo startDir)
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.Engine.Services.ExtensionService.StartService()
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.Engine.Services.ServiceManager.StartServices()
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.Engine.TestEngine.Initialize()
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.Engine.TestEngine.GetRunner(TestPackage package)
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.VisualStudio.TestAdapter.NUnitTestAdapter.GetRunnerFor(String assemblyName)
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyName, IFrameworkHandle frameworkHandle)
2016-02-23T14:45:49.6088663Z ##[error]   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
2016-02-23T14:45:49.6088663Z ##[error]
2016-02-23T14:45:49.6088663Z Information: NUnit Adapter 3.0.8.0 executing tests is finished

which correlates with %LOCALAPPDATA%\NUnit\Nunit30Settings.xml being accessed but might not be related. ExtensionService is here.

Issues #117 and #118 are related to this with a fix as provided by RobertK66:

If you install the Version 3.0.8.0 as a Nuget package then there is a corrupted (not empty!) “ignore.addins” file installed in the packages\NUnit3TestAdapter.3.0.8-ctp-8\lib folder. Earlier Versions did install a 0byte empty file!

After removing the content of this file in the /packages/NUnit3TestAdapter.3.0.8-ctp-8\lib folder and deleting the cached version of the adapter under \AppData\Local\Temp\VisualStudioTestExplorerExtensions... everything works as expected!