JMS queue polling example

Question

I'm trying to learn activemq + camel in order to apply it on a real world. I need to consume a queue, process the message and move it to another queue.

My concern regards to performance. I'll need to handle at least 100.000 messages daily. Right now I don't want to deal with vertical or horizontal scaling (we can't spent more money, until people gets convinced that the technology is good).

So, I thought about starting a few threads, that will poll the queue, consume, process and move the messages to other queue. The quantity of threads will depend on how the hardware responds to increasing levels of load.

My first question is: Is this a good approach (starting paralel threads to consume the queue)?

My second question is: I started my learning by reading Camel In Action. I don't know if I'm missing something, but I'm a little bit confused about how to build the consumer. By adapting FtpToJMSExample book example, I came out with the code bellow. In real world, I'll not create connections for each thread. I'll use a connection pool provided by the application server (glassfish).

public class JMSToJMSExample {
    public static void main(String args[]) throws Exception {
        CamelContext context = new DefaultCamelContext();
        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost");
        context.addComponent("jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));
        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("jms:in")
                  .process(new CustomProcessor())
                  .to("jms:out");
} });
        context.start();
        Thread.sleep(10000);
        context.stop();
    }
}

It works fine. But, the book calls it as a "polling" solution. I was expecting something like a while loop, so while the queue has messages, it keeps consuming. Ok, the example is polling a queue, but my point with the above example is that, if I reduce the sleep period, it will exit without processing all the messages that it could.

But anyway, I think that is better to establish a long time running thread, instead of asking the connection pool to give me a connection every time that the threads wake-up.

Please, since I'm learning, could you give some example of how to create a thread to poll a jms queue until it gets empty, not polling by time/period?

TIA,

Bob

Answer 1

1) use seda http://camel.apache.org/seda.html for concurrent processing

             from("jms:in")
             .to("seda:seda1");

             from("seda:sead1?concurrentConsumers=10")
             .process(new CustomProcessor())
             .to("jms:out");

2) read http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html about how to keep Camel running