Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dynamic addition of queues to a rabbit listener at runtime

I've got a project where we are going to have hundreds (potentially thousands) of queues in rabbit and each of these queues will need to be consumed by a pool of consumers.

In rabbit (using spring-amqp), you have the rabbitlistener annotation which allows me to statically assign the queues this particular consumer(s) will handle.

My question is - with rabbit and spring, is there a clean way for me to grab a section of queues (lets say queues that start with a-c) and then also listen for any queues that are created while the consumer is running.

Example (at start):

  • ant-queue
  • apple-queue
  • cat-queue

While consumer is running:

  • Add bat-queue

Here is the (very simple) code I currently have:

    @Component
    public class MessageConsumer {

        public MessageConsumer() {
            // ideally grab a section of queues here, initialize a parameter and give to the rabbitlistener annotation
        }

        @RabbitListener(queues= {"ant-queue", "apple-queue", "cat-queue"})
        public void processQueues(String messageAsJson) {
            < how do I update the queues declared in rabbit listener above ? >
        }
    }

Edit:

I should add - I've gone through the spring amqp documentation I found online and I haven't found anything outside of statically (either hardcoded or via properties) declaring the queues

like image 774
user10776719 Avatar asked Jan 08 '19 15:01

user10776719


People also ask

Does RabbitMQ automatically create queues?

In RabbitMQ, a producer never sends a message directly to a queue. Instead, it uses an exchange as a routing mediator. Therefore, the exchange decides if the message goes to one queue, to multiple queues, or is simply discarded.

Are RabbitMQ queues FIFO?

Queues in RabbitMQ are FIFO ("first in, first out"). Some queue features, namely priorities and requeueing by consumers, can affect the ordering as observed by consumers.

What is exclusive queue in RabbitMQ?

Exclusive queues are special queues which dispatch all messages to only one consumer at a time. This is useful when you want all messages to be processed serially but you can't or don't want to use Message Grouping.

What is listener in RabbitMQ?

The RabbitMQ Listener listens to RabbitMQ Queue for any incoming messages. The RabbitMQ Listener (Adapter) allows you to configure a RabbitMQ Listener from the Listener Type drop-down menu. Listener (Adapter) Configuration Drop-Down List.


1 Answers

  • Inject (@Autowired or otherwise) the RabbitListenerEndpointRegistry.

  • Get a reference to the listener container (use the id attribute on the annotation to give it a known id) (registry.getListenerContainer(id)).

  • Cast the container to an AbstractMessageListenerContainer and call addQueues() or addQueueNames().

Note that is more efficient to use a DirectMessageListenerContainer when adding queues dynamically; with a SimpleMessageListenerContainer the consumer(s) are stopped and restarted. With the direct container, each queue gets its own consumer(s).

See Choosing a container.

like image 179
Gary Russell Avatar answered Oct 22 '22 14:10

Gary Russell