Visual Studio 2012 debugging of remote process not working as expected

Question

I am struggling with a rather difficult debugging challenge and hoping that someone might have some clues how to make this work.

Here's the scenario:

I have a C# Windows service that runs under a user account with admin privileges and launches a separate executable process under a user account that has standard user privileges. The two processes are designed to communicate using WCF.

Unfortunately, when the child process is launched, it crashes immediately, with nothing in the event log that suggests what happened. The parent process continues running without exceptions.

For information: these two applications work reliably together in a configuration whereby the parent process is a desktop application. I have also had success with the parent as a Windows service, but only when both processes run under the same user account with admin privileges.

I now need to reconfigure their relationship to restrict the privileges of the child process, but this is when the crash occurs.

In order to prove that what I am trying to do is feasible, I have created two stub applications and launched them successfully in the desired configuration. So, I can deduce that my real child app contains something that is incompatible with this configuration and which causes a crash even before the code starts executing. Unfortunately, since the child process is a based on some rather complex legacy code, it is not easy to isolate its elements until I eliminate the problem, so I really need a reliable means of stepping through it.

If I modify the code of the child process to launch debugging immediately on startup, it invites me to attach a debugger, but fails to complete the attachment, with a message that indicates that The Just-in-time debugger does not have permission to debug the process.

I have also seen this question and attempted to implement this proposed solution (which looks really promising) but it fails to work in my scenario. Instead of launching debugging prior to launching the application it appears to do nothing - niether the debugger nor the application are launched and the debugging invite dialog is not displayed. However, I have verified that this technique works in my environment (by using it to launch Notepad.exe) so there is clearly something about my application or the way that I am launching it that is causing the problem.

I am happy to experiment and to share more details about my test results, if anyone has any suggestions.

Many thanks for your ideas,

Tim

Answer 1

According to my tests based on your scenario above (parent process is service with admin rights, child is console without admin rights), I see the same debugging error as you when I artificially force the child process to throw a permission exception as soon as it starts. The error message in this instance might be misleading, as it's not clear that this is actually a debugger permission problem

It would be useful to know what type of application your child process is, because that will affect the debugging options that you have.

The first way I've tried to debug this is to intercept all unhandled exceptions in my child process (a console app). You can do this by adding the following code in the start-up procedure of your child app:

AppDomain.CurrentDomain.UnhandledException += new 
    UnhandledExceptionEventHandler(App_UnhandledException);

Then I added code to my App_UnhandledException procedure to log the exception. This worked for me, and I could see the reason for the permission error. The only caveat is that this won't intercept an exception where your app won't even load because of the permission problem. But this approach should at least reduce your search space in understanding the permission problem.

If the exceptioon is generated before your exception handler is reached, another possiblity is to use the assembly binding log viewer. This is a very useful tool.

FWIW you can step through your service code (but unfortunately not into your child process) by starting your service within Visual Studio. The code shown below in the switch case called DEBUG will let you start/debug your service within VS.

// This is the entry point
static void Main(string[] args)
{
    // If parameter passed, act on it
    if ( args.Length > 0 )
    {
        switch (args[0] )
        {
            // Debug the service as a normal app from within Visual Studio
            case DEBUG:
                MyService DebugService = new MyService();
                DebugService.OnStart(null);
                break;
            // Install the service programatically
            case INSTALL:
                ManagedInstallerClass.InstallHelper(new string[] _
                { Assembly.GetExecutingAssembly().Location });
                break;
            // Un-install the service programatically
            case UNINSTALL:
                ManagedInstallerClass.InstallHelper(new string[] +
                { UNINSTALL, Assembly.GetExecutingAssembly().Location });
                break;
            // We don't understand this parameter!
            default:
                message = string.Concat(DEBUG, " to run service manually.",     Environment.NewLine);
                message += string.Concat(INSTALL, " to install service.",     Environment.NewLine);
                message += string.Concat(UNINSTALL, " to un-install service.",     Environment.NewLine);
                message += string.Concat("Do not understand the command-line parameter ", args[0]);
                throw new System.NotImplementedException(message);
        }
    }
    // If no parameter passed, just start the service normally
    else
    {
        ServiceBase[] ServicesToRun;
        ServicesToRun = new ServiceBase[] { new MyService() };
        ServiceBase.Run(ServicesToRun);
    }
}

Answer 2

The fact that the debugger never starts for the child means the error should be occuring in the PARENT server process. If you properly set the Image File Execution Options (which is easiest to do using GFlags program using the free Windows Debugging Tools from Microsoft), then that means you never start creating the child. The simplest way to test this is by adding an Assert to your code, right before the create child process call, build your parent service in debug mode, install/register it as a service, and start it up. When the Assert pops up, attach to the process, and start debugging from there. You should then see the create process error occuring in the parent.

If you want to interactively debug both the parent service and the child process, you can do this using WinDbg and GFlags, but it will be complicated.

You will need WinDbg and GFlags. These tools are included free from Microsoft as part of the Debugging Tools for Windows. You can find that free software package here: http://msdn.microsoft.com/en-us/windows/hardware/gg463009.aspx

Use GFlag to set the execution option for your PARENT SERVICE with the following debugger options:

"C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x86\WinDbg.exe" -server tcp:port=5000:9000 -o -g 

When Windows starts your parent service, it will do so under WinDbg. Because of the -o option, WinDbg will also manage the launched child process, allowing you to interactively DEBUG the child from start up. Because of the -g option, WinDbg will start the ParentService up and let it run, rather than stop it at load up as is normal debugging behavior. This will prevent Windows SCM from shutting it down and starting a new instance.

Because you are running a service, it will not have access to the desktop, so neither will its host WinDbg. You will have to attach ANOTHER debugger to the running instance of the WinDbg running your ParentService. You can do this using another WinDbg instance. To do that, start a second instance of WinDbg, and connect remotely by using the menu item "File|Connect To Remote Session...". At the dialog, put in:

tcp:Port=5000:9000,Server=[machinename]

Once you are connected, you will be able to work with your ParentService.exe, and when it creates the ChildProcess, the executing context will swap to it, and you will be able to debug it as well.

I've used this technique to debug a child process created by a windows service before. It isn't as easy as just debugging something in Visual Studio's built in debugger in its IDE, but it does work.

WinDbg has extensive documentation available for it, both from Microsoft and from other sources online. The URL I provided above includes links to WinDbg documentation.

I recommend using GFlags because it will make all the necessary edits to your Registry for running executables under a debugger of your choice. It also does much more, and is worth the time to learn about.

It is possible to set up breakpoints and set all sorts of options when WinDbg starts. I replace the -g option with the command line option:

-c "$$<c:\MyDebugCommands.txt"

This instructs WinDbg to run a command, and the command is to run a WinDbg script named "MyDebugCommands.txt". I populate MyDebugCommands.txt file with all the set up changes I need (such as load symbol options), as well as setting the breakpoints I am interested in, with the final command in the file being -g

As I said, it isn't as easy as just using the VS IDE and its built in debugger, but it will let you interactively debug your parent service and its launched child process.