Multithreaded .NET queue problems

Question

I'm have a wierd error in my code. It's extremely rare (happens once every few weeks maybe), but it's there and I'm not sure why.

We have 2 threads running, 1 thread gets networked messages and adds them to a Queue like this:

DataMessages.Enqueue(new DataMessage(client, msg));

Another thread takes messages off this queue and handles them, like this:

while (NetworkingClient.DataMessages.Count > 0)
{
    DataMessage message = NetworkingClient.DataMessages.Dequeue();

    switch (message.messageType)
    {
       ...
    }
}

However once every so often I get a NullReferenceException on the line switch (message.messageType) and I can see in the debugger that message is null.

It is not possible that a null value was put onto the queue (see the first bit of code), and these are the only 2 things that use the queue.

Is Queue not thread-safe, could it be that I'm dequeuing at the exact moment that the other thread is enqueuing and this causes the glitch?

Answer 1

Is Queue not thread-safe, could it be that I'm dequeuing at the exact moment that the other thread is enqueuing and this causes the glitch?

Exactly. Queue is not thread-safe. A thread-safe queue is System.Collections.Concurrent.ConcurrentQueue. Use it instead to fix your problem.

Answer 2

    while (NetworkingClient.DataMessages.Count > 0)
    {
        // once every two weeks a context switch happens to be here.
        DataMessage message = NetworkingClient.DataMessages.Dequeue();

        switch (message.messageType)
        {
           ...
        }
    }

... and when you get that context switch in that location, the result of the first expression (NetworkingClient.DataMessages.Count > 0) is true for both threads, and the one that get's to the Dequeue() operation first get's the object and the second thread get's a null (instead of InvalidOperationException because the Queue's internal state wasn't fully updated to throw the right exception).

Now you have two options:

1. Use the .NET 4.0 ConcurrentQueue

2. Refactor your code:

and make it look somehow like this:

while(true)
{
  DataMessage message = null;

  lock(NetworkingClient.DataMessages.SyncRoot) {
       if(NetworkingClient.DataMessages.Count > 0) {
          message = NetworkingClient.DataMessages.Dequeue();
       } else {
         break;
       }
    }
    // .. rest of your code
}

Edit: updated to reflect Heandel's comment.