CLR System.NullReferenceException when forcing 'Set Next Statement' into 'if' block

Question

Background

I accept this isn't something that can occur during normal code execution but I discovered it while debugging and thought it interesting to share.

I think this is caused by the JIT compiler, but would welcome any further thoughts.

I have replicated this issue targeting the 4.5 and 4.5.1 framework using VS2013:

---

Setup

To see this exception Common Language Runtime Exceptions must be enabled: DEBUG > Exceptions...

I have distilled the cause of the issue to the following example:

using System.Collections.Generic;
using System.Linq;

namespace ConsoleApplication6
{
    public class Program
    {
        static void Main()
        {
            var myEnum = MyEnum.Good;

            var list = new List<MyData>
            {
                new MyData{ Id = 1, Code = "1"},
                new MyData{ Id = 2, Code = "2"},
                new MyData{ Id = 3, Code = "3"}
            };

            // Evaluates to false
            if (myEnum == MyEnum.Bad) // BREAK POINT 
            {
                /*
                 * A first chance exception of type 'System.NullReferenceException' occurred in ConsoleApplication6.exe

                   Additional information: Object reference not set to an instance of an object.
                 */
                var x = new MyClass();

                MyData result;
                //// With this line the 'System.NullReferenceException' gets thrown in the line above:
                result = list.FirstOrDefault(r => r.Code == x.Code);

                //// But with this line, with 'x' not referenced, the code above runs ok:
                //result = list.FirstOrDefault(r => r.Code == "x.Code");
            }
        }
    }

    public enum MyEnum
    {
        Good,
        Bad
    }

    public class MyClass
    {
        public string Code { get; set; }
    }

    public class MyData
    {
        public int Id { get; set; }
        public string Code { get; set; }
    }
}

---

To Replicate

Place a breakpoint on if (myEnum == MyEnum.Bad) and run the code. When the break point is hit, Set Next Statement(Ctrl+Shift+F10) to be the opening brace of the if statement and run until:

Next, comment out the first lamda statement and comment in the second - so the MyClass instance isn't used. Rerun the process (hitting the break, forcing into the if statement and running). You'll see the code works correctly:

Finally, comment in the first lamda statement and comment out the second - so the MyClass instance is used. Then refactor the contents of the if statement into a new method:

using System.Collections.Generic;
using System.Linq;

namespace ConsoleApplication6
{
    public class Program
    {
        static void Main()
        {
            var myEnum = MyEnum.Good;

            var list = new List<MyData>
            {
                new MyData{ Id = 1, Code = "1"},
                new MyData{ Id = 2, Code = "2"},
                new MyData{ Id = 3, Code = "3"}
            };

            // Evaluates to false
            if (myEnum == MyEnum.Bad) // BREAK POINT 
            {
                MyMethod(list);
            }
        }

        private static void MyMethod(List<MyData> list)
        {
            // When the code is in this method, it works fine
            var x = new MyClass();

            MyData result;

            result = list.FirstOrDefault(r => r.Code == x.Code);
        }
    }

    public enum MyEnum
    {
        Good,
        Bad
    }

    public class MyClass
    {
        public string Code { get; set; }
    }

    public class MyData
    {
        public int Id { get; set; }
        public string Code { get; set; }
    }
}

Rerun the test and everything works correctly:

---

Conclusion?

My assumption is the JIT compiler has optimized out the lamda to always be null, and some further optimized code is running prior to the instance being initialized.

As I previously mentioned this could never happen in production code, but I would be interested to know what was happening.

Answer 1

This is a pretty inevitable mishap, not related to optimization. By using the Set Next Statement command, you are bypassing more code than you can easily see from the source code. It only becomes obvious when you look at the generated machine code. Use Debug + Windows + Disassembly at the breakpoint. You'll see:

            // Evaluates to false
            if (myEnum == MyEnum.Bad) // BREAK POINT 
0000016c  cmp         dword ptr [ebp-3Ch],1 
00000170  setne       al 
00000173  movzx       eax,al 
00000176  mov         dword ptr [ebp-5Ch],eax 
00000179  cmp         dword ptr [ebp-5Ch],0 
0000017d  jne         00000209 
00000183  mov         ecx,2B02C6Ch               // <== You are bypassing this
00000188  call        FFD6FAE0 
0000018d  mov         dword ptr [ebp-7Ch],eax 
00000190  mov         ecx,dword ptr [ebp-7Ch] 
00000193  call        FFF0A190 
00000198  mov         eax,dword ptr [ebp-7Ch] 
0000019b  mov         dword ptr [ebp-48h],eax 
            {
0000019e  nop 
                /*
                 * A first chance exception of type 'System.NullReferenceException' occurred in ConsoleApplication6.exe

                   Additional information: Object reference not set to an instance of an object.
                 */
                var x = new MyClass();
0000019f  mov         ecx,2B02D04h             // And skipped to this
000001a4  call        FFD6FAE0 
// etc...

So, what is that mysterious code? It isn't anything you wrote in your program explicitly. You can find out by using the Set Next Statement command in the Disassembly window. Move it to address 00000183, the first executable code after the if() statement. Start stepping, you'll see it executing the constructor of a class named ConsoleApplication1.Program.<>c__DisplayClass5

Otherwise well covered in existing SO questions, this is an auto-generated class for the lambda expression in your source code. It is required to store captured variables, list in your program. Since you skipped its creation, dereferencing list in the lambda is always going to bomb with NRE.

A standard case of a "leaky abstraction", C# has some of it but not outrageously so. Nothing much you can do about it of course, you can certainly blame the debugger for not guessing at this correctly but it is a very difficult problem to solve. It cannot easily find out if that code belongs to the if() statement or the code that follows it. A design issue, debug info is line number based and there is no line of code. Also in general a problem with the x64 jitter, it fumbles even in simple cases. Which should be fixed in VS2015.

This is something you have to learn the Hard Way™. If it is really, really important then I showed you how to set the next statement properly, you have to do it in the Disassembly view to make it work. Feel free to report this issue at connect.microsoft.com, I'd be surprised if they didn't already know about it however.