Visual Studio debugger stops hitting breakpoints in mixed debugging mode

Question

I have a severe problem with mixed debugging in MSVC2013. After a call to a COM method from a native C++ DLL, debugger does not stop at breakpoints any more.

Code structure

The picture above presents the overall structure of the code.

I have a single solution with about ten C# projects, about fifty C++ native projects, and one C++/CLI project serving as a bridge between managed and native worlds. Startup project is a C# WPF project (GUI Application), it calls C++/cli project (Bridge) internally, which in turn calls various native C++ Dlls (Various libraries). Alternatively, I can make a C++ console application (Service console app) as startup project for testing purposes only.

I have implemented a library to import some information from Autodesk Inventor document files. Inventor Apprentice COM server (Inventor Apprentice on picture) is used to achieve it, which was freely downloaded alongside with Inventor View 2015. As a first step, the import was implemented in a standalone native C++ console application, and everything worked fine. Then it was adapted to be used in the whole infrastructure as a native C++ dll (Import library), and the debugging hell started.

Symptoms

"debugging broken". In a debug build, after calling the following COM method in Import library:

auto pComponentDefinitions = pDocument->GetComponentDefinitions();

breakpoints in C++ code are no longer hit. Even if I set a breakpoint in the code of another DLL, it is not hit. Breakpoints still visualize as full red circles, so this is not related to PDB issues. The application itself continues to execute, and some time later I can see the correct results of data import visible in GUI, meaning that Import library has executed correctly. After that, I can pause the GUI Application with Break All button, in which case the main thread is shown stuck deeply inside one of the Inventor's dlls (rse.dll), which cannot be true because that thread has finished import and has even returned correct results.

In the Output window, I can see the following messages, appearing during the problematic COM method call (access violations seem to be normal in Apprentice):

First-chance exception at 0x000007FEDD451F0C (rse.dll) in GUIApplication.exe: 0xC0000005: Access violation writing location 0x000007FFFDE3AFCC.
The Common Language Runtime cannot stop at this exception. Common causes include: incorrect COM interop marshalling and memory corruption. To investigate further use native-only debugging.
First-chance exception at 0x000007FEDD455F6C (rse.dll) in GUIApplication.exe: 0xC0000005: Access violation writing location 0x000007FFFDE3EE6C.

I have tried to embed breakpoints into the code at the moment of compilation, by inserting __debugbreak() before and after the problematic import code. The first one is hit (if debugging is not yet broken), but the second one is not. On the other hand, debugger clearly notices it, because it writes the following message to Output window:

The process hit a breakpoint the Common Language Runtime cannot continue from.
This may be caused by an embedded breakpoint in the native runtime or a breakpoint set in a can't-stop region.
To investigate further, use native-only debugging.

Google gives no results at all for this diagnostic message. It sounds like MSVC thinks it is debugging managed code, which is actually native.

"crashes in call". In case of release build, running the application in mixed debugging mode results in a crash inside rse.dll during the problematic COM call.

Reproducibility

I use MSVC 2013 update 4. Project is built in x64 mode. Net Framework v4.0 is used. Inventor Apprentice from Inventor 2015 is used. Experimentation shows that:

1. Everything is OK when no debugger is attached.
2. Everything works correctly (including debugging), when native-only debugging is used (either via Service console app or after attaching to already running process in native only mode).
3. In mixed (i.e. native + managed) debugging mode the problem reproduces regardless of whether GUI application was started with debugging or debugger was attached to a working process.
4. There is a problem both in debug and release modes, but it acts differently. In debug build crazy debugging issues arise ("debugging broken"), but in release it simply crashes somewhere inside ("crashes inside").

Full list of runs performed can be seen here.

The main question

Has anyone seen similar behavior before? What may be the cause of such behavior? Is there a way to fix it?

Answer 1

Disabling the new managed debugging engine of MSVC has helped to fix the issue. It can be done by going to Tools > Options > Debugging > General > tick "Use Managed Compatibility Mode".

---

While trying to find a workaround for the issue, I have posted the following question. Hans Passant has posted not only a workaround, but also a solution to my main problem. It seems that new debugging engine is not properly working in case of C++/CLI interop.

P.S. Given that the symptoms are quite unique and crazy, I decided to post the full question with the answer, hoping that this information may help some people with similar problems in future.