Clustering in Apache Camel: Multiple JVMs Same CamelContext

Question

I have one application that we deploy on a cluster. The cluster might have 2 or 4 JVMs as per the environment. The application has same CamelContext which we are deploying on all the JVMs. Hence, all the JVMs have same routes. For FTP routes, that is good as it makes it competitive and only 1 JVM gets the files. However, while using timer based operation to fetch from DB, I see that all the JVMs read the same set of records and do the same job. What I want is, if one route picks it up, the other routes shouldn't try. I tried googling for this. However, couldn't find the best way to do it. Is there any default camel component which supports this? I read the clustering and load balancing on Camel Documentation, but it didn't help. JGroups and ZooKeeper are specific to type of clusters. Any help would be appreciated.

Answer 1

First, the different deployed Camel contexts will act as standalone applications with no knowledge of each other.

The behavior if you have multiple routes consuming from the same sources is very component dependent.

• File/FTP and similar typically have file locking mechanisms to avoid multiple consumers reading the same file.
• Message queueing (JMS/AMQP/etc) has built in handling of messages, unless you're using topics - then each instance will get a copy.
• Database and other pollable components might need you to trigger the polling using some kind of signal that is generated from only one node.

I guess the last one is your primary question. It can be done by several means. They are probably a bit tricky to setup, but should do the job.

• A clustered Quartz scheduler. Typically requires a database that is to be shared. Read here how to configure quartz in Camel. Then you need to configure quartz.properites and the clustering properties (JDBC or whatnot). That is documented here.
• You can have a single route running (elected among your deployed Camel instances) using ZooKeeper and route policies. Then you can either use that route and fail over to an other route in case of problems, or use that single route to emit triggers, using timer or quartz, to poll that can be distributed to all camel instances.