How to properly unit test a .NET project with multiple target frameworks, given implementation differences among targets?

Question

Consider a .NET class library that targets the following frameworks:

• .NET Framework 2.0
• .NET Framework 4.6
• .NET Standard 1.0
• .NET Standard 1.3
• .NET Portable Profile336

Let's not immediately worry about why this is the exact list of target frameworks.

Supposing that the external API across all target frameworks is identical, but each individual target framework has some code that's unique to that framework, how do I properly write the unit tests that cover the code from all target frameworks?

I'm running Visual Studio 2017 Community.

I've cooked up a ready-to-go minimal sample on GitHub here with what I've found so far to be probably the best I can do right now: https://github.com/airbreather/MultiTargetDemo, but this doesn't feel like the right way to do it. Some flaws:

• Requires a separate .csproj file for each target framework with lots of copy-paste.
• Uses a not-well-documented* property called ReferringTargetFrameworkForProjectReferences.
  • *that's a bit of an understatement... Googling it at the moment, literally nothing shows up apart from Microsoft's SDK stuff.
• Doesn't seem to work properly with the .NET Standard shims (.NET Standard 1.3 here throws at runtime trying to look for System.IO.FileSystem). (edit: figured it out, this was corefx issue #16322, which has a bit of an awkward workaround).

The library in the above-linked has precisely one public class with precisely one public instance method on it. The method is contractually obligated to just return the 32-bit integer value 3, but it computes that 3 a materially different way on each target. Obviously the actual use case revolves around implementing much more "interesting" methods, but this is sufficient for conversation.

Furthermore, note that the test class in that library actually uses some stuff on its own that's not available on all target platforms: System.Threading.Tasks.Task<TResult> is not available on .NET Framework 2.0... this serves as a proxy for just anything that the test itself might be using for its own purposes, which might not necessarily be available in the target framework that's being tested.

The actual use case is NetTopologySuite, which already targets multiple full-framework platforms and some PCLs; there's a year-old issue asking to make it work with .NET Core. I've been working on-and-off to try to make that happen for just about a month now, and I'm finally stumbling into this issue.

I found that other question: Unit testing code targeting several frameworks at once with .NET Core and xUnit, which is indeed similar to this one. This is different for a few reasons:

• That question was using the VS2015-era preview SDK tooling (project.json). This question refers to the VS2017-era non-preview tooling with the pretty .csproj files and all.
• That question could be resolved by simply multi-targeting the test project itself with one "test" target framework per framework targeted by the "subject-under-test". It does not appear that this question can be resolved in a similar way, especially with the unusual Profile336 target.
  • Besides, given that the full .NET Framework 4.6 can reference assemblies targeted at any of the listed five target frameworks, there should be no need for me to have to limit the implementations of the fundamental test classes to just what's available in the intersection of whatever grab-bag of target frameworks that get selected for this.
  • I did try a hack like listing synthetic stuff in TargetFrameworks like test1;test2;test3;test4;test5 and then forcing TargetFrameworkIdentifier + TargetFrameworkVersion to .NETFramework + 4.6, but I didn't have much success (it failed when trying to grab the xunit package). I also didn't try too hard on this, however.

Answer 1

First, test projects can multi-target as well by making use of the <TargetFrameworks>property just like you used for the library. While the VS test runner currently only shows/runs one framework (the first one to be specific), any invocation of dotnet test will execute all frameworks (xunit is also developing custom console runner - dotnet xunit - for a better console UX).

Also keep in mind that test projects need to specify a target framework that has a runtime. So your test projects can either target netcoreapp* or net* but never netstandard* since it does not have a runtime associated to run the tests. I'm not sure about the portable target since the Microsoft.NET.Tests.Sdk that ever test project needs to reference to get test runner support doesn't have conditions for NETPortable.

So you'd need to pick all the versions of net and netcoreapp that you need to test on / support - e.g. net20;net45;net46;netcoreapp1.0 would cover all your .net framework and .net core versions.

There is a way to force a dependency to a specific target framework by setting a Properties="TargetFramework=netstandard1.3" on the <ProjectReference> element, but this may be hard to get working correctly for restore and multi-targeting.

Answer 2

Just had this issue. Turns out, as you cannot test with .net standard, I had to target the specific framework versions in the test project, .net5, 4.6.2 etc Then when running the tests, they will run against those versions to as with the library that is targeting those other versions.

Our test coverage was showing 94% for .net5 and zero for the others till we did this.