Retrying messages at some point in the future (ActiveMQ)

Question

I'm working on a system in ActiveMQ where I would really prefer not to lose messages. My problem is that retrying messages is causing my consumers to block (instead of working on messages they could handle). I'd like to give failed messages several days to retry (for example, one of my potential destinations is another server I'll access via SFTP, which could be down), but I don't want a consumer blocking for several days -- I want it to keep working on the other messages.

Is there a way to tell the broker to resend the message later? Right now I'm looking into taking the message off of the queue and putting it on with a delay, but I'm wondering if there's a simpler way. I'm using Apache Camel, so a solution using that would be good too.

Answer 1

Camel can definitely help with this...

One way is to use a separate queue and periodically retry messages separately from the main flow (especially when performance is a concern). Also, this provides a separate queue to allow you to triage those error messages (view, clear, alter, manually retry, etc)...

something like this...see polling consumer for more details

//main route to process message from a queue (needs to be fast)
from("activemq:queue:mainQ").process(...);

//handle any errors by simply moving them to an error queue (for retry later)
onException(Exception.class)
    .handled(true).to("activemq:queue:mainErrorQ");

//retry the error queue
from("timer://retryTimer?fixedRate=true&period=60000")
    .bean(myBean, "retryErrors"); 

...

public void retryErrors() {
    // loop to empty queue
    while (true) {
        // receive the message from the queue, wait at most 3 sec
        Exchange msg = consumer.receive("activemq:queue.mainErrorQ", 3000);
        if (msg == null) {
            // no more messages in queue
            break;
        }

        // send it to the starting queue
        producer.send("activemq:queue.mainQ", msg);
    }
}   

If you land on a better solution, let me know...good luck