Strange behaviour of C# in debugger vs normal execution caused a Heisenbug

Question

I just spent the last hour tackling a bizarre issue with unmanaged memory in C#.

First, a bit of context. I've got a C# DLL which exports some native methods (via this awesome project template) which are then called by a Delphi application. One of these C# methods has to pass back a struct to Delphi, where it's cast to a record. Already I can tell you're feeling nauseous, so I won't go into any more detail. Yes, it's ugly, but the alternative is COM... no thanks.

Here's a simplification of the offending code:

IntPtr AllocBlock(int bufferSize)
{
    IntPtr ptrToMem = Marshal.AllocHGlobal(bufferSize);

    // zero memory
    for(int i = 0; i < bufferSize; i++)
       Marshal.WriteInt16(ptrToMem, i, 0);

    return ptrToMem;
}

In reality there's some other stuff going off in here, related to native resource tracking, but that's basically it. If you already spotted the bug, well done.

Essentially, the problem was that I was using WriteInt16 instead of WriteByte, due to an IntelliSense-aided typo, resulting in the final iteration writing one byte over the end of the buffer. Easy mistake to make, I guess.

However, what made this such a pain in the proverbial to debug was that it failed silently within the debugger, and the remainder of the application continued to work. The memory had been allocated, and all but the last byte was zero'd, so it worked fine. When launched outside a debugger, it caused the application to crash with an access violation. A classic Heisenbug situation - the bug disappears when you try to analyse it. Note that this crash wasn't a managed exception, but rather a real CPU-level access violation.

Now, this baffles me for two reasons:

1. The exception was NOT thrown when any debugger was attached - I tried Visual Studio, CodeGear Delphi 2009 and OllyDbg. When attached, the program worked perfectly. When not attached, the program crashed. I used exactly the same executable file for all attempts. My understanding is that debugging should not change the behaviour of the application, yet it clearly did.

2. Normally I'd expect this operation to cause an AccessViolationException within my managed code, but instead it died with a memory access violation in ntdll.dll.

Now, fair enough, my case is probably one of the most obscure (and possibly misguided) corner-cases in the history of C#, but I'm lost as to how attaching any debugger prevented the crash. I'm especially surprised that it worked under OllyDbg, which doesn't interfere with the process anywhere near as much as Visual Studio would.

So, what the heck happened here? Why was the exception swallowed (or not raised) during debugging, yet not when outside the debugger? And why wasn't a managed access violation exception thrown when I tried to call Marshal.WriteInt16 outside the allocated memory block, as the documentation says it should?

Answer 1

You just set some heap memory to zero. This does not fail as there are no checks where you got this address from. This creates a problem in the heap management (in ntdll) later on. This page shows why a overrun is only detected later on.

You are not failing with attached debugger for similar reasons. When windows detects that a debugger is attached, a different heap manager is used. The debug heap manager adds a suffix for detecting overruns, that does not disable the heap manager but shows the corruption on freeing the block. See the page above on more details.

How the heap manager is switched on os level is not known to me but i found a related answer on stackoverflow: Visual C++: Difference between Start with/without debugging in Release mode

As i understand the heap manager switching, if you start the process in release mode from the desktop/console and attach a debugger later on, the debugger should stop on the heap corruption as the standard heap manager is used. This could be used as a test for my assumptions.