Handling exceptions in BlockingCollection

Question

I have a blocking BlockingCollection that I use to process a queue (i.e. sending emails).

BlockingCollection<ItemQueue> queue = new BlockingCollection<ItemQueue>();
Thread threadQueue = new Thread(ProcessQueue);
threadQueue.Start();

private void ProcessQueue()
{
    while (true)
    {
        var item = queue.Take();

        process(item); 
    }
}

public void process(ItemQueue item)
{
    try
    {
        // do something
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

Basically I run a separate thread where I wait for an element in the queue (somewhere there is an Add() call). Then I process that item. This function might raise an exception: if we're talking about emails, for example, the Send() may fails.

I would like to handle this situation in two different ways, depending on the specific application:

1. repeat the process function with the same item until it completes without any exception
2. put this item at the end of the queue in order to try again later, but process the other items now

Is there a better approach that handles these behaviors natively?

Answer 1

Both approaches have their shortcomings:

• The first approach may stall the queue indefinitely in case an item has permanent delivery problems
• The second approach will keep an item with permanent delivery problems in the queue indefinitely

A better approach would establish a separate blocking collection for items that failed, push them into that collection on exception, along with the exception object, and forget about them. A separate thread would take these items off the queue, analyze the exception, and decide what to do next. It could schedule re-sending at an appropriate time, push the item back if its failure count is under some fixed number, file the message for analysis by administrators, or simply discard it, depending on the needs of your system.