Why will Visual Studio 2019 will not run my unit tests?

Question

I am seeing some very strange behavior for NUnit tests in VS2019, where the same solution works fine in VS2017. There are several NUnit test projects in my soultion.

In VS2017 with the NUnit Runner extension installed, I can see all of my tests in the Test Explorer window and the "run all" button will work and run all tests. Some developers in my organization use Resharper instead of the NUnit extension and this works too.

I have stopped using Resharper because as VS introduces more features, Resharper has made it so slow that VS is unusable.

In VS2019 the Test Explorer window will show all of my unit tests (even without the NUnit extension installed). If I click "run all" it will not run any tests and the Output window will say 0 tests discovered. Coworkers have said that Resharper will run all tests without issue. If I right click a single test project and run just those tests, some projects will run tests, but not all.

For some projects I have tried installing the NUnit3TestAdapater nuget package and this will let VS2019 run that project's test if only that project is selected. This does not work for all projects and it still does not work for "run all".

Does anyone know what might be causing this and what could fix it? I have updated to the latest version of NUnit (3.12) and latest TestAdapter (3.16) for all of these projects.

This has been a real pain for about a year, because I need to keep VS2017 and 2019 both installed, and I need to train new developers on how to work around this weird issue.

Answer 1

The solution turned out to be a combination of two things.

1. On the top menu, navigating to Test > Options and disabling "Discover tests in real time from C# and Visual Basic .NET source files". This option appears to be incompatible with parameterized tests. The problem is that parameterized tests do not "exist" in source code but are generated by the test adapter at runtime. This also fixed an issue I've seen where the "base" test of the parameterized test shows up in the Test Explorer as a not-run test, even though only the specific test cases are real tests. (In my opinion, this should not be enabled by default because parameterized tests are extremely useful, whereas seeing new tests in the Test Explorer without compiling is a trivial convenience since you must compile to run them anyway.)

2. Using the NuGet package for the test adapter instead of using the VSIX extension. It seems that for this piece, all that is required is that at least one project in your solution references this. If at least one project references it, all test projects can be run. (This makes sense to me, as it is more compatible with build tools outside of Visual Studio.)

Hooray for breaking changes!