"Analyzer with Code Fix" project template is broken

Question

Short question: How to setup a roslyn code analyzer project with a working unit-test project in Visual Studio 2019 v16.6.2?

A few months (and a few Visual Studio updates) ago I experimented with setting up a code analyzer project using the "Analyzer with Code Fix (.NET Standard)" project template. It worked well and just as documented in all the available how-tos.

Today (VS2019 v16.6.2) I wanted to start a real analyzer project, but unfortunatly the updated template seems to be broken or to be released with a lot of unfinished work in progress. (One minor issue is the fact that suddenly the package manager seems unable to restore packages, because it does not like the vsix project using the same assembly name as the analyzer project.)

The template contains a unit-test project. In the earlier version I experimented with, this test project contained a lot of code that acted like a test infrastructure to make it easy for developers to test the analyzer against real code.

All this code now seems to be integrated in a lot of language-specific nuget packages. But these packages

• are no longer hosted in https://dotnet.myget.org/F/roslyn/api/v3/index.json, but in https://dotnet.myget.org/F/roslyn-analyzers/api/v3/index.json instead
• they are all pre-release versions and
• they do not contain the expected classes/code.

These are the package references apparently required (ommitted test framework packages):

• Microsoft.CodeAnalysis.CSharp.Analyzer.Testing.MSTest v1.0.1-beta1.20384.1
• Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.MSTest v1.0.1-beta1.20384.1
• Microsoft.CodeAnalysis.CSharp.CodeRefactoring.Testing.MSTest v1.0.1-beta1.20384.1

And this is the sample unit-test code:

using System.Threading.Tasks;
using Microsoft.VisualStudio.TestTools.UnitTesting;

using Verify = Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.MSTest.CodeFixVerifier<
    Analyzer1.Analyzer1Analyzer,
    Analyzer1.Analyzer1CodeFixProvider>;

namespace Analyzer1.Test
{
    [TestClass]
    public class UnitTest
    {
        //No diagnostics expected to show up
        [TestMethod]
        public async Task TestMethod1()
        {
            var test = @"";

            await Verify.VerifyCSharpDiagnosticAsync(test);
        }

        //...

I managed to get the packages installed by adding the correct source url, and I updated all referenced packages to the latest (pre-release) version. But whatever I tried, this template code could not be compiled, because the namespace

Microsoft.CodeAnalysis.CSharp.CodeFix

(used in the alias declaration of Verify) could not be found. Googling for that namespace only brought me back to the myget.org sites for the Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.* packages.

---

So, what do I have to do to setup a working unit-test (ideally mstest) project for my code-analyzer project? I would also be fine using the "old" version that included all the helper code directly in the project, not using the apparently unfinished nuget packages.

---

UPDATE: After implementing AndrewSilvers's answer this second test from the template fails:

    [TestMethod]
    public async Task TestMethod2()
    {
        var test = @"
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Diagnostics;

namespace ConsoleApplication1
{
    class TypeName
    {   
    }
}";

        var fixtest = @"
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Diagnostics;

namespace ConsoleApplication1
{
    class TYPENAME
    {   
    }
}";

        var expected = Verify.Diagnostic("Analyzer1").WithLocation(11, 15).WithArguments("TypeName");
        await Verify.VerifyCodeFixAsync(test, expected, fixtest);
    }
}

stating that one diagnostic was found, but none expected (which is obviously wrong, the test clearly expects a diagnostic).

So the question remains: How to fix this test project?

Answer 1

I found a couple of repositories with working unit tests:

• https://github.com/dotnet/samples/tree/master/csharp/roslyn-sdk/Tutorials/MakeConst
• https://github.com/dotnet/roslyn/tree/master/src/Analyzers/CSharp

All of them seem to be using "manual" approach with helper code included in the project. While they don't shed any light on what's going on with project templates and helper code bundled into "beta" nugets, at least they provide a working starting point.

UPDATE: Microsoft has updated its documentation; "Build your first analyzer and code fix" tutorial now has a note in Prerequisites section explaining that there's a bug in the template that should be fixed in Visual Studio v16.7. The note also provides steps for fixing generated projects until then.

UPDATE 2: Followed the steps in the note. In fact, for test sources they are the same as provided by AndrewSilver in his answer. Got the same result, the second test fails. Oh well, back to square one.

UPDATE 3: Ok, I think I got it. Short version: add the following line to your analyzer's Initialize method:

context.ConfigureGeneratedCodeAnalysis(GeneratedCodeAnalysisFlags.None);

Long version

Spent some time debugging the library. The main culprit appeared to be this line that would declare your expected diagnostic message a subject to exclusion and discard it from the list of expected results, thus failing the test due to mismatch between actual (1) and expected (now 0) diagnostics. Was about to file an issue but decided to check if anything similar already exists. It does, indeed, and it points to another lengthy thread with this solution.

By the way, project's README file provides a few more usage examples.