Designing a Java OSGi application using RabbitMQ while using PreparedStatement

Question

I am designing an application that will run in an OSGi container (currently Equinox). It will receive messages over RabbitMQ and process them internally. The application will run continually as a server. My current plan is to have the RabbitMQ listener bundle configure its queues and place listeners on them, using QueueingConsumer and running in their own threads. The listeners would call one or more processing services to handle the messages. The processors need to make JDBC calls to access databases. I would like to be able to control the order in which the processors are called. It would be nice to have the flexibility to add more services at a later time without having to recode the RabbitMQ listeners.

The problem I face is that messages may come in bursts or slowly. I would like to be able to use PreparedStatement to speed up the database access, but I also do not want to hold the connections open long term while nothing is going on. I have thought about subclassing DefaultConsumer directly and letting it run on the threads in the RabbitMQ Connection, but then I lose the ability to know when nothing is going on. My original idea was to keep the message processors completely separate as OSGi services, and have each grab a database connection from a pool each time it was called, but that loses the advantage of prepared statements. I'm using the Tomcat JDBC pool, and it does not seem to have prepared statement caching. Also, I'm not sure how expensive creating a prepared statement for every call would be, but it seems wasteful.

The best idea that I have come up with so far is to have my listeners process in a double loop. The outer loop waits for a message, then calls an inner loop that establishes database connections and prepared statements and runs until no more messages come in for a specified timeout, then closes its connections and returns to the outer loop. I got this to work for a single bit of processing, but I am having trouble visualizing how to manage this if I have multiple processors that may have different prepared statements.

Maybe I have to give up the idea of multiple services and hard code the processing into my listeners.

Any suggestions? Thanks!

Answer 1

Why not use services as listeners and pass them the JDBC connection they should use? A single piece of code then dispatches the queues to your services. This central dispatcher can trivially maintain a pool of prepared JDBC connections. If you do not want to see the JDBC connection in the API, use the Coordinator service to hold a connection. "Aware" services can then get the optimized JDBC connection.

Alternatively, register a DataSource as a service for your listener service and implement your own pool policy. Since you know the caller in an OSGi service call, you can do all kinds of optimizations, e.g. read the prepared statements from the bundle's jar and cache them accordingly.

I would not give up on services here, the decoupling you get is great, I known from experience. Since in this model the service listeners have no dependency on RabbitMQ you can easily test them and/or switch to another queue technology.

Then again, in these cases the best thing is to do the most simple solution you can come up with. If you have a performance problem, measure, and fix the bottlenecks. An insane amount of effort is wasted on premature solutions ...I can tell you quite a few dumb premature optimizations I've wasted my time on.