Roslyn Analyzer Rule does not fail the build

Question

Following on from this tutorial from MS, I have created an analyzer for Roslyn.

According to the page, you can mark the rule as DiagnosticSeverity.Error, and this will cause the build to break:

In the line declaring the Rule field, you can also update the severity of the diagnostics you’ll be producing to be errors rather than warnings. If the regex string doesn’t parse, the Match method will definitely throw an exception at run time, and you should block the build as you would for a C# compiler error. Change the rule’s severity to DiagnosticSeverity.Error:

internal static DiagnosticDescriptor Rule = new DiagnosticDescriptor(DiagnosticId, Title, MessageFormat, Category, DiagnosticSeverity.Error, isEnabledByDefault: true, description: Description);

In my code, I have created the rule more or less as detailed here:

private static readonly DiagnosticDescriptor Rule = 
  new DiagnosticDescriptor(DiagnosticId, Title, MessageFormat, Category,
  DiagnosticSeverity.Error, true, helpLinkUri: HelpUrl);

This rule works fine. It throws up the red lines, it displays the message in the errors list. However, the build succeeds, and I am able to successfully run the application.

NB: I've created this rule to capture Thread.Sleep for this example.

Is there additional setup required to ensure a rule breaks the build?

Answer 1

This is a feature of Analyzers running from a VSIX file.

If the IDE-installed rules ran as part of the in-IDE build, it would result in IDE builds and command line builds having potentially very different outputs. For example, a user with code-cracker installed as a VSIX could end up filing a bug report that an open source project does not build due to an analyzer error (or perhaps a warning when the project uses /warnaserror). They would be forced to either uninstall the analyzer extension or modify the rule set used by the project to disable some rule that only exists on one developer's machine.

In contrast, rules that are installed via NuGet become part of the project and part of the build. They run the same way across developer machines, and they run the same way in-IDE, on the command line, and in automated build environments.

Source: IDE rules don't fail builds

In order to make the build fail for the rules, you need to add the analyzer as a nuget package to the project. This will ensure that failures will cause the build to fail as expected.