Access violation in code that is not mine

Question

I'm not sure how to go about debugging this. I have a C# program consisting entirely of managed code, running in .NET 4.5. After running it for a while, at some seemingly random time, I get a an error "An unhandled exception of type 'System.AccessViolationException' occurred in mscorlib.dll". Since I'm running it from Visual Studio (2012) I click "break" and am presented with the following call stack:

mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) + 0x47 bytes     [Native to Managed Transition]   ntdll.dll!_NtRequestWaitReplyPort@12()  + 0xc bytes  kernel32.dll!_ConsoleClientCallServer@16()  + 0x4f bytes     kernel32.dll!_GetConsoleLangId@4()  + 0x2b bytes     kernel32.dll!_SetTEBLangID@0()  + 0xf bytes  KernelBase.dll!_GetModuleHandleForUnicodeString@4()  + 0x22 bytes    mdnsNSP.dll!7177aa48()   [Frames below may be incorrect and/or missing, no symbols loaded for mdnsNSP.dll]    mdnsNSP.dll!71775b06()   mdnsNSP.dll!71774ded()   mdnsNSP.dll!71774e8c()   bcryptprimitives.dll!746d1159()      bcryptprimitives.dll!746d1137()      ntdll.dll!_LdrpCallInitRoutine@16()  + 0x14 bytes    ntdll.dll!_NtTestAlert@0()  + 0xc bytes  ntdll.dll!_NtContinue@8()  + 0xc bytes   ntdll.dll!_LdrInitializeThunk@8()  + 0x1a bytes  

An interesting thing I notice is that nothing in the call stack is my code.

What strategy would you advise I use to find the route of the problem? Or have you seen a problem similar to this and have any tips?

Since the exception doesn't seem to include my code, I don't know what information to include that would be helpful in answering the question, but ask me if there is anything else that I should include.

Since the error may be IO related (since PerformIOCompletionCallback is at the top of the stack), this is a list of typical IO tasks that this application performs:

• TcpListener.AcceptTcpClientAsync
• NetworkStream.Write/BeginRead/EndRead
• SqlCommand.BeginExecuteReader/EndExecuteReader
• StreamWriter.WriteLine

Other notes:

• It seems to be roughly repeatable - I get the same error in the same place (PerformIOCompletionCallback), but have to wait a different length of time to get it (in the order of minutes).
• I don't think I can manufacture a small program that reliably highlights the problem. My program handles many thousands of similar IO operations before it hits this error.

Edit:

Based on the suggestion by @Kevin that Mdnsnsp.dll is from Bonjour, I uninstalled Bonjour and tried again. The exception persists, but the call stack is much cleaner:

mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) + 0x47 bytes     [Native to Managed Transition]   kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes     ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes    ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes     

I'm assuming the Bonjour installer installed some benign hook DLL for network traffic, but uninstalling it did not fix the problem.

Edit:

I have temporarily re-coded all my unsafe functions using slower "safe" equivalents to eliminate that as a suspect. Now none of the assemblies in the application are compiled using unsafe switch. The problem still persists. To reiterate, I now have no unsafe code, no native code and no P/Invoke calls (in user code) in this application, but I am still experiencing the AccessViolationException as described above.

Answer 1

Since its PerformIOCompletionCallback that is causing the error I would look at your Asynchronous IO calls.

• TcpListener.AcceptTcpClientAsync
• NetworkStream.Write/BeginRead/EndRead
• SqlCommand.BeginExecuteReader/EndExecuteReader

The error looks to be happening because the handle that was registered is no longer valid. Since it is happening in Managed code the cause is going to be from a Managed object and NOT from a 3rd party native DLL.

Answer 2

don't know if it can help you but looks like we faced a similar problem several years ago. As I remember our investigation pointed on dll in other program - we found that memory access violations can be caused by antiviruses (NOD32 in our case), firewalls or network sniffers/traffic controllers.

Try to check applications log (Control Panel -> System and Security -> Administrative Tools -> Event Viewer) for errors caused by above applications. If problem is with other program try to disable/uninstall it and check again if crash still appears in your programm.

UPD Have you tried to reproduce this issue on the clean test environment?