Does RabbitMQ call the callback function for a consumer when it has some message for it?

Question

Does RabbitMQ call the callback function for a consumer when it has some message for it, or does the consumer have to poll the RabbitMQ client?

So on the consumer side, if there is a PHP script, can RabbitMQ call it and pass the message/parameters to it. e.g. if rating is submitted on shard 1 and the aggregateRating table is on shard 2, then would RabbitMQ consumer on shard 2 trigger the script say aggRating.php and pass the parameters that were inserted in shard 1?

Answer 1

The AMQPQueue::consume method is now a "proper" implementation of basic.consume as of version 1.0 of the PHP AMQP library (http://www.php.net/manual/en/amqpqueue.consume.php). Unfortunately, since PHP is a single threaded language, you cant do other things while waiting for a message in the same process space. If you call AMQPQueue::consume and pass it a callback, your entire application will block and wait for the next message to be sent by the broker, at which point it will call the provided callback function. If you want a non blocking method, you will have to use AMQPQueue::get (http://www.php.net/manual/en/amqpqueue.get.php), which will poll the server for a message, and return a boolean FALSE if there is no message.

I disagree with scvatex's suggestion to use a separate language for using a "push" approach to this problem though. PHP is not IO driven, and therefore using a separate language to call a PHP script when a message arrives seems like unnecessary complexity: why not just use AMQPQueue::consume and let the process block (wait for a message), and either put all the logic in the callback or make the callback run a separate PHP script.

We have done the latter at my work as a large scale job processing system so that we can segregate errors and keep the parent job processor running no matter what happens in the children. If you would like a detailed description of how we set this up and some code samples, I would be more than happy to post them.

Answer 2

What you want is basic.consume, which allows the broker to push messages to clients.

That said, the libraries are implemented differently. Most of them have support for basic.consume, but because of inherent limitations of the frameworks used, some don't (most notably the official RabbitMQ C client on which a lot of other clients are based).

If your PHP library does not support basic.consume, you either have to use polling (bad), or you could use one of the more complete clients to drive the script. For instance, you could write a Python or Java program that consumes from the broker (so, the broker pushes deliveries to them) and they could call the script whenever a new message is received. The official tutorials are a great introduction to the AMQP APIs and are a good place to start.

This is efficient from most points of view, but it does require a stable connection to the broker.

If in doubt about the capabilities of the various clients, or if you need more guidance, the RabbitMQ Discuss mailing list is a great place to ask questions. The developers make a point of answering any query posted there.