CreateProcess with new console window, but override some std i/o handles

Question

If you use CreateProcess with the flag CREATE_NEW_CONSOLE, the new process has its standard input, output, and error handles directed to the new console window. If you want to override the I/O streams, you can do so by setting the handles in STARTUPINFO fields hStdOutput, hStdInput, and hStdError and setting the flag STARTF_USESTDHANDLES.

But what if you want to override only one of the handles? For example, I might want to redirect stderr to a file while leaving the stdout and stdin connected to the new console window.

The STARTF_USESTDHANDLES flag tells CreateProcess to replace all of the handles instead of connecting them to the ones for the new console window. So it seems we must provide all three handles. Obviously I can set hStdError to the handle of the log file, but what values should be used for hStdInput and hStdOutput?

I tried using NULL, which seems to work on Windows 8.1, but it doesn't work on Windows 7.

I also thought about creating a console window first, and then calling CreateProcess with handles to the new console window's buffers (and omitting the CREATE_NEW_CONSOLE flag). Unfortunately, the parent process is also a console application, and it seems a console application cannot create a second console window.

Answer 1

According to this MSDN Support article:

How to spawn console processes with redirected standard handles

If the parent process only wishes to redirect one or two standard handles, specifying GetStdHandle() for the specific handles causes the child to create the standard handle as it normally would without redirection. For example, if the parent process only needs to redirect the standard output and error of the child process, then the hStdInput member of the STARTUPINFO structure is filled as follows:

hStdInput = GetStdHandle(STD_INPUT_HANDLE);

According to the GetStdHandle() documentation:

STD_INPUT_HANDLE
(DWORD)-10
The standard input device. Initially, this is the console input buffer, CONIN$.

STD_OUTPUT_HANDLE
(DWORD)-11
The standard output device. Initially, this is the active console screen buffer, CONOUT$.

STD_ERROR_HANDLE
(DWORD)-12
The standard error device. Initially, this is the active console screen buffer, CONOUT$.

...

The standard handles of a process may be redirected by a call to SetStdHandle, in which case GetStdHandle returns the redirected handle. If the standard handles have been redirected, you can specify the CONIN$ value in a call to the CreateFile function to get a handle to a console's input buffer. Similarly, you can specify the CONOUT$ value to get a handle to a console's active screen buffer.

Attach/detach behavior

When attaching to a new console, standard handles are always replaced with console handles unless STARTF_USESTDHANDLES was specified during process creation.

If the existing value of the standard handle is NULL, or the existing value of the standard handle looks like a console pseudohandle, the handle is replaced with a console handle.

When a parent uses both CREATE_NEW_CONSOLE and STARTF_USESTDHANDLES to create a console process, standard handles will not be replaced unless the existing value of the standard handle is NULL or a console pseudohandle.

So, if the parent process's STDIN has NOT been redirected, GetStdHandle(STD_INPUT_HANDLE) will return either NULL or a pseudo-handle that refers to CONIN$. When that value is passed to the child process via STARTUPINFO, the child process will receive a console handle for the STDIN of whichever console it happens to be running in. On the other hand, if the parent process's STDIN has been redirected, GetStdHandle(STD_INPUT_HANDLE) will return an actual handle to a real file/pipe/etc, which the child will inherit and access.

The same applies to STDOUT and STDERR.

So, if you want to redirect the child's STDIN/OUT/ERR handles, you have to set hStdInput/Output/Error to your own handles. If you want the child to receive default handles, use GetStdHandle() and let CreateProcess() decide what kind of handles the child receives based on whether the parent is itself being redirected or not.

Answer 2

On Windows 7, at least, the documentation here (as quoted by the existing answers) is misleading. The recommended approach only works if the parent process does not have redirected input.

This is the actual behaviour, as observed on my machine:

• When a process opens a handle to CONOUT$ (for example) the returned handle always has the same numeric value (on my machine, the handle for CONOUT$ is always 7) unless that handle already exists.

• So if you are a console process, and you were not launched with redirected output, your standard output handle is 7. If you open another handle to CONOUT$, it will have a different value. If you close handle 7, and then open a handle to CONOUT$, you'll get 7 again.

• When a console process is launched, a handle to CONOUT$ with the magic value of 7 is usually present in the child process (regardless of redirection). If you have used STARTF_USESTDHANDLES, then the child's standard output will go to the console if and only if you specify 7 as the standard output handle. If you specify a handle to CONOUT$ that has any other value, it won't work.

• Sometimes when a console process is launched with standard output redirected, there is no handle to CONOUT$ present in the child process. In particular, this happens when cmd.exe launches a console process with standard output redirected. So far, I have been unable to figure out what combination of parameters to use with CreateProcess to make this happen.

• The behaviour of CONIN$ is analogous; on my machine, the magic value for CONIN$ is 3. (I haven't investigated how standard error works.)

The upshot is that I do not believe it is safe to make use of this behaviour unless you have complete control over the way the parent process was launched, because unless the standard handles already point to the console there is no reliable way of obtaining a handle to the console with the correct magic value.

Instead, launch a proxy process with CREATE_NEW_CONSOLE and without STARTF_USESTDHANDLES. From that process, since you know how it was launched, you know the standard handles will have the correct magic values, so it is safe to specify them as the handles for a child process.

Answer 3

If i interpreted this documentation correctly, you should use GetStdHandle(STD_INPUT_HANDLE) and GetStdHandle(STD_OUTPUT_HANDLE) for the other two handles.