In a FIFO Qeueing system, what's the best way the to implement priority messaging

Question

For message-oriented middleware that does not consistently support priority messages (such as AMQP) what is the best way to implement priority consumption when queues have only FIFO semantics? The general use case would be a system in which consumers receive messages of a higher priority before messages of a lower priority when a large backlog of messages exists in Queue(s).

Answer 1

Given only FIFO support for a given single queue, you will of course have to introduce either multiple queues, an intermediary, or have a more complex consumer.

Multiple queues could be handled in a couple of ways. The producer and consumer could agree to have two queues between them, one for high-priority, and one for background tasks.

If your producer is constrained to a single queue, but you have control over the consumer, consider introducing a fan-out router in the path. So producer->Router is a single queue, and the router then has two queues to the consumer.

Another way to tackle it, which is likely less than ideal, would be to have your consumer spin a thread to front the queue, then dispatch the work internally. Something like the router version above, but inside a single app. This has the downside of having multiple messages in flight inside your app, which may complicate recovery in the event of a failure.

Don't forget to consider starvation of the effectively low-priority events, whatever they are, if some of them should be processed even if there are higher-priority events still hanging about.