Roslyn analyzer only runs for open files

Question

Backstory (not essential to understand for my problem, but some context may help)

At my company, we use an IResult<T> type to handle errors in a functional style, rather than throwing exceptions and hoping some client catches them. A IResult<T> can either be a DataResult<T> with a T or an ErrorResult<T> with an Error but not both. Error is roughly equivalent to an Exception. So a typical function would return IResult<T> to pass any encountered errors through the return value, rather than back up the stack with throw.

We have extension methods on IResult<T> to compose function chains. The two main ones are Bind and Let.

Bind is your standard monadic bind operator from functional languages. Basically if the IResult has a value, it projects the value, otherwise it forwards the error. It is implemented as such

static IResult<T2> Bind(
    this IResult<T1> @this, 
    Func<T1, IResult<T2>> projection)
{
    return @this.HasValue 
        ? projection(@this.Value) 
        : new ErrorResult<T2>(@this.Error);
}

Let is used for executing side effects, as long as an error was not encountered earlier in the function chain. It is implemented as

static IResult<T> Let(
    this IResult<T> @this,
    Action<T> action) 
{
    if (@this.HasValue) {
        action(@this.Value);
    }
    return @this;
}

---

My use case for a Roslyn analyzer

A common mistake that is made when using this IResult<T> API, is to call a function that returns an IResult<T> inside the Action<T> passed into Let. When this happens, if the inner function returns an Error, the error is lost and execution continues like nothing went wrong. This can be a very hard bug to track down when it happens, and it has happened several times in the past year. In these situations, Bind should be used instead, so that the error can be propagated.

I want to identify any calls to functions that return IResult<T> inside lambda expressions passed as arguments to Let and flag them as compiler warnings.

I have created an analyzer to do this. You can check out the full source code here: https://github.com/JamesFaix/NContext.Analyzers Here is the main analyzer file in the solution:

using System.Collections.Immutable;
using System.Linq;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.Diagnostics;

namespace NContext.Analyzers
{
    [DiagnosticAnalyzer(LanguageNames.CSharp)]
    public class BindInsteadOfLetAnalyzer : DiagnosticAnalyzer
    {
        public const string DiagnosticId = "NContext_0001";
        private const string _Category = "Safety";
        private const string _Title = "Unsafe use of IServiceResponse<T> inside Let expression";
        private const string _MessageFormat = "Unsafe use of IServiceResponse<T> inside Let expression.";
        private const string _Description = "If calling any methods that return IServiceResponse<T>, use Bind instead of Let. " + 
            "Otherwise, any returned ErrorResponses will be lost, execution will continue as if no error occurred, and no error will be logged.";

        private static DiagnosticDescriptor _Rule = 
            new DiagnosticDescriptor(
                DiagnosticId, 
                _Title, 
                _MessageFormat,
                _Category, 
                DiagnosticSeverity.Warning, 
                isEnabledByDefault: true, 
                description: _Description);

        public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics { get; } =
            ImmutableArray.Create(_Rule);

        public override void Initialize(AnalysisContext context)
        { 
            context.RegisterSyntaxNodeAction(AnalyzeNode, SyntaxKind.InvocationExpression);
        }

        private static void AnalyzeNode(SyntaxNodeAnalysisContext context)
        {
            var functionChain = (InvocationExpressionSyntax) context.Node;

            //When invoking an extension method, the first child node should be a MemberAccessExpression
            var memberAccess = functionChain.ChildNodes().First() as MemberAccessExpressionSyntax;
            if (memberAccess == null)
            {
                return;
            }

            //When invoking an extension method, the last child node of the member access should be an IdentifierName
            var letIdentifier = memberAccess.ChildNodes().Last() as IdentifierNameSyntax;
            if (letIdentifier == null)
            {
                return;
            }

            //Ignore method invocations that do not have "Let" in the name
            if (!letIdentifier.GetText().ToString().Contains("Let")) 
            {
               return;
            }

            var semanticModel = context.SemanticModel;

            var unsafeNestedInvocations = functionChain.ArgumentList
                //Get nested method calls
                .DescendantNodes().OfType<InvocationExpressionSyntax>()
                //Get any identifier names in those calls
                .SelectMany(node => node.DescendantNodes().OfType<IdentifierNameSyntax>())
                //Get tuples of syntax nodes and the methods they refer to
                .Select(node => new 
                {
                    Node = node,
                    Symbol = semanticModel.GetSymbolInfo(node).Symbol as IMethodSymbol
                })
                //Ignore identifiers that do not refer to methods
                .Where(x => x.Symbol != null
                //Ignore methods that do not have "IServiceResponse" in the return type
                    && x.Symbol.ReturnType.ToDisplayString().Contains("IServiceResponse"));

            //Just report the first one to reduce error log clutter
            var firstUnsafe = unsafeNestedInvocations.FirstOrDefault();
            if (firstUnsafe != null)
            {
                var diagnostic = Diagnostic.Create(_Rule, firstUnsafe.Node.GetLocation(), firstUnsafe.Node.GetText().ToString());
                context.ReportDiagnostic(diagnostic);
            }
        }
    }
}

---

Problem

My analyzer works fine for any *.cs files that are currently open. Warnings are added to the Errors window and the green warning underline shows up in the text editor. However, if I close a file that contains callsites with these warnings, the errors are removed from the Errors window. Also, if I just compile my solution with no files opened, no warnings are logged. When running the analyzer solution in debug mode, no breakpoints are hit when there are no source code files open in the debugging sandbox instance of Visual Studio.

How can I make my analyzer check all files, even closed ones?

Answer 1

I found an answer from this question: How can I make my code diagnostic syntax node action work on closed files?

Apparently, if your analyzer is installed as a Visual Studio extension, rather than as a project-level package, it defaults to only analyzing open files. You can go to Tools > Options > Text Editor > C# > Advanced and check Enable full solution analysis to have it work for any file in the current solution.

What a strange default behavior. ¯\_(ツ)_/¯