How do I use WaitForMultipleObjects to balance competing work?

Question

I'm using WaitForMultipleObjects in an IPC situation, where I have one process writing data to either or both of two memory mapped files and another process that picks up that data as it's updated. I'm using named event objects to notify the second process when data in either of the MMFs has changed. There's also an event for terminating the "watcher" thread.

So a reduced example of the code is something like this (EDIT - note that the event objects have been created as auto-reset events)

unsigned int CMyClass::ThreadFunc()
{
    // background thread
    HANDLE hEvent[3];

    // open events for updates 0 and 1 and kill signal
    hEvent[0] = ::OpenEvent(SYNCHRONIZE, FALSE, _T("KillEvent"));
    hEvent[1] = ::OpenEvent(SYNCHRONIZE, FALSE, _T("UpdateEvent0"));
    hEvent[2] = ::OpenEvent(SYNCHRONIZE, FALSE, _T("UpdateEvent1"));

    // main loop
    while (true)
    {
        // wait for any event and break on kill signal
        DWORD dwRet = WaitForMultipleObjects(3, hEvent, FALSE, INFINITE);
        if (dwRet == WAIT_OBJECT_0) break;

        // which update event did we get?
        if (dwRet == WAIT_OBJECT_0 + 1) 
        {
            // perform update from channel 0
        }
        else if (dwRet == WAIT_OBJECT_0 + 2)  
        {
            // perform update from channel 1
        }
    }

    // release handles 
    for (int i = 0; i < 3; ++i)
        CloseHandle(hEvent[i]);

    // exit thread
    return 0;
}

In the most common use case, only one of the MMFs is updated, so this code works fine. However, when both MMFs are being updated, so I get two events signalled, I noticed through logging and debugging that the first event was being processed roughly twice as often as the second event - even though the process performing the updates was just calling SetEvent on each of them in adjacent lines of code. This gave the appearance of one update being slower than the other and hence a bug report from a user.

Looking closer at MSDN, it indicates why this might be happening

If multiple objects become signaled, the function returns the index of the first handle in the array whose object was signaled.

So it seems like the second event is only breaking the wait if the processing in the code above manages to finish executing before another SetEvent gets called on the first event.

So, to temporarily workaround the problem, I just unilaterally perform both updates, regardless of which event was set.

        // wait for any event
        DWORD dwRet = WaitForMultipleObjects(3, hEvent, FALSE, INFINITE);
        if (dwRet == WAIT_OBJECT_0) break;

        // perform update from channel 0

        // perform update from channel 1

This is obviously not ideal and it's very wasteful because, like I said above, for the most common use case, only one MMF is being updated.

What is the best way to handle this type of situation? I considered using two threads - one for each MMF and corresponding event - but the "update" code is common to both and would involve adding a lot of synchronisation that currently is unnecessary.

Do I have any other options?

Answer 1

After you process one event you can rearrange the array of handles passed to WaitForMultipleObjects in the next call. So completing event 1 makes event 2 the priority event the next time around. And vice-versa.

Answer 2

If the return value from WaitForMultipleObjects is WAIT_OBJECT_1 then you can still check if the event for MMF 2 is set using WaitForSingleObject.

DWORD dwRet = WaitForMultipleObjects(3, hEvent, FALSE, INFINITE);

if (dwRet == WAIT_OBJECT_0) break;

// decide which MMFs need processing
if ( dwRet == WAIT_OBJECT_1 )
{
    if ( WaitForSingleObject( hEvent[2], 0 ) == WAIT_OBJECT_0 )
        // both MMFs have been updated, decide which to do first.
    else
        // only MMF 1 has been updated, process that
}
else if ( dwRet == WAIT_OBJECT_2 )
{
    // only MMF 2 has been updated as WaitForMultipleObjects returns 
    // lowest index of set event.
}

// do processing