In what cases does the Process.Start() method return false?

Question

From MSDN:

The return value true indicates that a new process resource was started. If the process resource specified by the FileName member of the StartInfo property is already running on the computer, no additional process resource is started. Instead, the running process resource is reused and false is returned.

Trying something like this:

var info = new ProcessStartInfo {FileName = @"CMD"};

var p1 = new Process
{
     StartInfo = info
};

var result = p1.Start(); //true
result = p1.Start(); //true

var p2 = new Process
{
    StartInfo = info
};

result = p2.Start(); //true

Have the same result if I'm using FilePath = @"c:\myapp.exe" instead of CMD.

In what cases does it return false?

Answer 1

If you take a look at the reference source, you'll see how Process.Start works:

http://referencesource.microsoft.com/System/R/c50d8ac0eb7bc0d6.html

That is one of two methods called when you call Process.Start. Notice near the bottom where it returns the value true or false. False is only returned if, after starting the process, it cannot obtain the handle for the process that was started.

In order to start the process, it uses NativeMethods.CreateProcess which you can find the source of here: http://referencesource.microsoft.com/System/compmod/microsoft/win32/NativeMethods.cs.html#9c52a5ca5f3eeea3

Which is just the method prototype for Kernel32.CreateProcess, which the API is found here: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx

If you look at the return values, it says:

If the function succeeds, the return value is nonzero. If the function fails, the return value is zero.

I can't find anything in the API for CreateProcess that says it fails if the requested process is already running, perhaps if the process failed to start because it is a single instance application (like Outlook) then it may fail, but for multiple instance applications like the command prompt, it shouldn't fail to create a handle to the process.

So, after saying all that, it is possible that the MSDN documentation is not entirely correct, I don't have the link you have, but for the Process.Start(StartInfo), MSDN says this about the return value:

A new Process that is associated with the process resource, or null if no process resource is started. Note that a new process that’s started alongside already running instances of the same process will be independent from the others. In addition, Start may return a non-null Process with its HasExited property already set to true. In this case, the started process may have activated an existing instance of itself and then exited.

(Emphasis added by me). It doesn't say the call will fail if its already running.

For Process.Start(), it says:

Return Value Type: System.Boolean true if a process resource is started; false if no new process resource is started (for example, if an existing process is reused).

So it says if an existing process is reused, this is entirely up to the application being started or the method of starting it.

Answer 2

You can technically get a false return when you use ProcessStartInfo.UseShellExecute = true (the default) and you launch the process by passing a document filename. And the shell is somehow able to figure out to pass the document open request to an already running instance of the process.

The only documented case of this is opening a web page in Internet Explorer. There might be other ones, probably having something to do with legacy DDE activation. That's a guess.

This is otherwise a specific case of a general problem with Process.Start(), there are lots of single-instance apps around. The Office apps as the most common example. The most typical behavior you'd observe is that the process very quickly terminates again. It just detected that the app was already running and used process-interop to ask the running instance to open the document. The kind of feature also supported in .NET.

You'll have no idea which specific process is showing the document unless you know it is a single-instance app and its process name so you have some hope of finding it back with Process.GetProcessesByName(). This is however not fail-safe, there might be an unrelated process running that happens to have the same name. Advantage of getting false is that you know there's no point in waiting for it to terminate.