Overlapped ReadFileEx on Child Process' Redirected STDOUT Never Fires

Question

I have a long-running console-based application Sender that sends simple text to STDOUT using non-buffered output such as cout << "Message" << flush(). I want to create an MFC dialog-based application (named Receiver) that starts Sender and can read it's output. Receiver should also be able to detect when Sender has died, or be able to kill Sender if it wants to. Sender knows nothing of Reciever, and I can't change Sender's code.

I have asked a separate question about the best way to do this. My first attempt was to create pipes with redirected STDIN and STDOUT for the child process and use asynchronous ReadFileEx calls to read in Sender's data. This isn't working correctly, because the ReadFileEx function only fires once, and only with zero bytes transferred even though I know for a fact that Sender is sending data.

I am creating 2 pipes with redirected STDIN and STDOUT, ala this MS example:

// allow the child process to inherit handles
SECURITY_ATTRIBUTES sa = {0};
sa.nLength = sizeof(sa);
sa.bInheritHandle = 1;

// create pipes with rerouted stdin & stdout
CreatePipe(&handles[h_Child_StdOut_Read], &handles[h_Child_StdOut_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdOut_Read], HANDLE_FLAG_INHERIT, 0);
CreatePipe(&handles[h_Child_StdIn_Read], &handles[h_Child_StdIn_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdIn_Read], HANDLE_FLAG_INHERIT, 0);

...Receiver then goes on to start Sender via CreateProcess():

// create child process
PROCESS_INFORMATION pi = {0};
STARTUPINFO si = {0};
si.cb = sizeof(si);
si.hStdOutput = handles[h_Child_StdOut_Write];
si.hStdInput = handles[h_Child_StdIn_Read];
si.dwFlags |= STARTF_USESTDHANDLES;
CreateProcess( 0, "Sender.EXE", 0, 0, 1, 0, 0, 0, &si, &pi);
handles[h_Child_Process] = pi.hProcess;
handles[h_Child_Thread] = pi.hThread;

My main loop is based on WaitForObjectsEx, placed in to an alertable wait state to support the asynch file read. I am waiting on two handles: one that signals when Sender dies prematurely, and one that signals when Receiver's main thread wants Sender to die. Before starting the loop, I kick off an overlapped (asynchronous) file read operation on Sender's STDOUT. Ignore the obvious memory leaks and other hacks -- this is illustrative:

vector<HANDLE> wait_handles;
wait_handles.push_back(handles[h_Die_Sig]);
wait_handles.push_back(handles[h_Child_Process]);

for( bool cont = true; cont; )
{
    IO* io = new IO;
    memset(io, 0, sizeof(IO));
    io->buf_size_ = 16 * 1024;
    io->buf_ = new char[io->buf_size_];
    memset(io->buf_, 0, io->buf_size_);
    io->thread_ = &param;
    io->file_ = handles[h_Child_StdOut_Read];
    if( !ReadFileEx(io->file_, io->buf_, io->buf_size_, io, OnFileRead) )
    {
        DWORD err = GetLastError();
        string err_msg = util::strprintwinerr(err);
    }

    DWORD rc = WaitForMultipleObjectsEx(wait_handles.size(), &wait_handles[0], FALSE, INFINITE, TRUE);

    // ...
}

The IO object above is derived publicly from OVERLAPPED:

struct IO : public OVERLAPPED
{
    char* buf_;
    DWORD buf_size_;
    DWORD read_;
    ThreadParam* thread_;
    HANDLE file_;
};

When the overlapped Read function completes, I read the incoming data and generate a string:

void CALLBACK OnFileRead(DWORD err, DWORD bytes, OVERLAPPED* ovr)
{
    IO* io = static_cast<IO*>(ovr);
    string msg(io->buf_, bytes);
}

Sender knows nothing of Receiver, and it sends text to the console using very simple, but non-buffered means.

The problem: I know that Sender is sending data to its STDOUT, but my OnFileRead function is called only once, and only with zero bytes transferred.

Why can't I receive Sender's output this way? Do I have a bug, or am I doing something wrong?

Answer 1

Besides the error pointed out by @DyP, you are assuming that CreatePipe opened the handle in overlapped mode. Your assumption is incorrect. Microsoft documents it:

Asynchronous (overlapped) read and write operations are not supported by anonymous pipes. This means that you cannot use the ReadFileEx and WriteFileEx functions with anonymous pipes. In addition, the lpOverlapped parameter of ReadFile and WriteFile is ignored when these functions are used with anonymous pipes.

(Indeed, if you look inside kernel32.dll, on Windows XP for example, CreatePipe does not set the lower bit on the seventh parameter to NtCreateNamedPipeFile; that bit is set when CreateNamedPipe is called with FILE_FLAG_OVERLAPPED.)

Look for Dave Hart's MyCreatePipeEx implementation; it can be used as a drop-in replacement for CreatePipe when overlapped I/O is needed. Simply change PipeSerialNumber++ to InterlockedIncrement(&PipeSerialNumber) to avoid race conditions in MT code.

Answer 2

I think you have a typo:

CreatePipe(&handles[h_Child_StdOut_Read], &handles[h_Child_StdOut_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdOut_Read], HANDLE_FLAG_INHERIT, 0);
CreatePipe(&handles[h_Child_StdIn_Read], &handles[h_Child_StdIn_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdIn_Read], HANDLE_FLAG_INHERIT, 0);

change the last one to

SetHandleInformation(handles[h_Child_StdIn_Write], HANDLE_FLAG_INHERIT, 0);

that's also what they do at the MSDN example.