Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

RabbitMQ: dropping messages when no consumers are connected

Tags:

rabbitmq

amqp

I'm trying to setup RabbitMQ in a model where there is only one producer and one consumer, and where messages sent by the producer are delivered to the consumer only if the consumer is connected, but dropped if the consumer is not present.

Basically I want the queue to drop all the messages it receives when no consumer is connected to it.

An additional constraint is that the queue must be declared on the RabbitMQ server side, and must not be explicitly created by the consumer or the producer.

Is that possible?

I've looked at a few things, but I can't seem to make it work:

  1. durable vs non-durable does not work, because it is only useful when the broker restarts. I need the same effect but on a connection.
  2. setting auto_delete to true on the queue means that my client can never connect to this queue again.
  3. x-message-ttl and max-length make it possible to lose message even when there is a consumer connected.
  4. I've looked at topic exchanges, but as far as I can tell, these only affect the routing of messages between the exchange and the queue based on the message content, and can't take into account whether or not a queue has connected consumers.

The effect that I'm looking for would be something like auto_delete on disconnect, and auto_create on connect. Is there a mechanism in rabbitmq that lets me do that?

like image 985
LordOfThePigs Avatar asked Jul 26 '17 13:07

LordOfThePigs


People also ask

Can RabbitMQ lose messages?

In order to avoid losing messages on the RabbitMQ (as opposed to application) side, queues and messages must be able to cope with RabbitMQ node restarts, node and hardware failures. With some messaging protocols supported by RabbitMQ, applications control durability of queues and messages.

Why are messages lost when I restart a RabbitMQ broker?

Messages, exchanges, and queues that are not durable and persistent will be lost during a broker restart. If you cannot afford to lose any messages, make sure that your queue is declared as durable and your messages are sent with delivery mode "persistent".

Does RabbitMQ create queue if not exists?

If queue or exchange doesn't exist then it will throw error. if exists it will not do anything. Actually, this is the correct answer. the one that has been chosen as correct, emits a potential risk of declaring and creating abandoned Queues in RabbitMQ server.

How does RabbitMQ push messages to consumers?

RabbitMQ sends acknowledgements to publishers on message receipt. Consumers maintain persistent TCP connections with RabbitMQ and declare which queue(s) they consume. RabbitMQ pushes messages to consumers. Consumers send acknowledgements of success/failure.


1 Answers

After a bit more research, I discovered that one of the assumptions in my question regarding x-message-ttl was wrong. I overlooked a single sentence from the RabbitMQ documentation:

Setting the TTL to 0 causes messages to be expired upon reaching a queue unless they can be delivered to a consumer immediately

https://www.rabbitmq.com/ttl.html

It turns out that the simplest solution is to set x-message-ttl to 0 on my queue.

like image 128
LordOfThePigs Avatar answered Oct 24 '22 08:10

LordOfThePigs