Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How long a RabbitMQ Message stays alive without Subscribers?

I am creating a simple Publisher/Subscriber using MassTransit and RabbitMQ. The Publisher has the following code to initialize the bus:

/** create the bus */
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
    var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
    {
        h.Username("guest");
        h.Password("guest");
    });
});

/** start the bus and publish */
bus.Start();
bus.Publish<IPersonLogin>(new {FirstName = "John", LastName = "Smith"});

And the Subscriber has this code for initialization:

var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
    var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
    {
        h.Username("guest");
        h.Password("guest");
    });

    cfg.ReceiveEndpoint(host, "person_login", e =>
    {
        e.Consumer<PersonLoginConsumer>();
    });
});

If I shut-down the Subscriber and publish 2 messages, the messages are not lost and as soon as the Subscriber comes back to life the Messages are processed.

So my questions are:

  1. How do I ensure that a Message stays in the Queue of RabbitMQ until one Subscriber comes up and pick it up?
  2. What happen if the Server is reboot and some Messages were not processed by any Subscriber, do they get lost or do they get processed as soon as the Subscriber come alive after reboot?
  3. Is this the correct pattern to ensure that every single message is processed or should I use a different strategy?
like image 374
Raffaeu Avatar asked Jan 12 '16 15:01

Raffaeu


2 Answers

by default, any message sitting in a queue will remain there until one of three things happens:

  1. the message is consumed
  2. the message "time to live" expires (default is to live forever)
  3. the server crashes or restarts

if you have a queue full of messages, the messages will generally stick around until one of those three things happens. hopefully you will have your consumers online soon enough that you can consume the messages and process them.

you would only set a time to live (ttl) if you want messages to be automatically deleted after a period of time (assuming they are not consumed first)

for crashes... a message can survive a crash / restart if you make the message persistent to disk. there's still a chance that the message will be lost if the server crashes before the message is routed from the exchange to the queue, though.

like image 134
Derick Bailey Avatar answered Oct 29 '22 18:10

Derick Bailey


On top of mind.

  1. If there arent any subscribers RabbitMQ wont know to which queue a message should be delivered. Then a message will be undeliverable.(Not sure if this will be moved to a error queue or skipped)
  2. If the exchanges are already there it will be placed in the queue of consumer that has subscribed to the event. So you endpoint hosting your consumer can be down the message will still be delivered.
  3. When the message is delivered to the queue the consumer will pick up your message and process it. If a exception occurs while processing your message it will be moved to the endpoint_error queue. (Depending on your RetryPolicy). Deploy a fix and move you message back in to the main queue and the messages will be processed as if nothing has happend.

Good read for common issues on common gotcha's

  • Under the Hood
  • Common Gotcha's
like image 38
Maarten Mensink Avatar answered Oct 29 '22 16:10

Maarten Mensink