What is the equivalent of System.Diagnostics.Debugger.Launch() in unmanaged code?

Question

I need to launch a debugger from my native C++ program when certain conditions are met. In C# I just call System.Diagnostics.Debugger.Launch(). I thought that Win32 DebugBreak() call will do what I want, but it just terminates the application if there is no debugger present.

How do I launch a new instance of the debugger (the famous "Possible debuggers" dialog) from native code? Is it even possible? I could try to use COM to create a new instance of Visual Studio, but it is kinda complicated, and will also lock me to a particular version of VS.

Answer 1

I turns out that it is possible to call vsjitdebugger.exe directly with the PID of the current process. Make sure that "Native" is selected in Tools->Options->Debugging->Just-in-Time in Visual Studio.

Here's C++ code to launch debugger. It uses UNICODE versions of various Win32 APIs. I get System directory, because CreateProcess() does not use PATH.

bool launchDebugger()
{
    // Get System directory, typically c:\windows\system32
    std::wstring systemDir(MAX_PATH+1, '\0');
    UINT nChars = GetSystemDirectoryW(&systemDir[0], systemDir.length());
    if (nChars == 0) return false; // failed to get system directory
    systemDir.resize(nChars);

    // Get process ID and create the command line
    DWORD pid = GetCurrentProcessId();
    std::wostringstream s;
    s << systemDir << L"\\vsjitdebugger.exe -p " << pid;
    std::wstring cmdLine = s.str();

    // Start debugger process
    STARTUPINFOW si;
    ZeroMemory(&si, sizeof(si));
    si.cb = sizeof(si);

    PROCESS_INFORMATION pi;
    ZeroMemory(&pi, sizeof(pi));

    if (!CreateProcessW(NULL, &cmdLine[0], NULL, NULL, FALSE, 0, NULL, NULL, &si, &pi)) return false;

    // Close debugger process handles to eliminate resource leak
    CloseHandle(pi.hThread);
    CloseHandle(pi.hProcess);

    // Wait for the debugger to attach
    while (!IsDebuggerPresent()) Sleep(100);

    // Stop execution so the debugger can take over
    DebugBreak();
    return true;
}

Answer 2

DebugBreak() is fine, so is the __debugbreak() intrinsic. They both do the same thing, they crash the program with a STATUS_BREAKPOINT exception. Which then triggers the Windows Error Reporting dialog, it trundles for a while then offers the Debug button. Which then starts the debugger.

The only real mistake you could make is not waiting long enough for the WER dialog and pressing Cancel too quick. Or having WER disabled. If there is no debugger available at all then, yes, you don't get to choose one.

The registry key that matters is HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug\Debugger. Normally set to vsjitdebugger.exe, the one that displays the "Possible debuggers" dialog.