Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using publisher confirms with RabbitMQ, in which cases publisher will be notified about success/failure?

Quoting the book, RabbitMQ in Depth:

A Basic.Ack request is sent to a publisher when a message that it has published has been directly consumed by consumer applications on all queues it was routed to or that the message was enqueued and persisted if requested.

Confused with Has been directly consumed, does it mean when consumer send ack to broker publisher will be informed that consumer process message successfully? or it means that publisher will be notified when consumer just receive message from the queue?

or that the message was enqueued and persisted if requested. Is this like conjuction or publisher will be informed when either of those happens? (In that case publisher would be notified twice)

Using node.js and amqplib wanted to check what is happening actually:

// consumer.js
amqp.connect(...)
.then(connection => connection.createChannel())
.then(() => { assert exchange here })
.then(() => { assert queue here })
.then(() => { bind queue and exchange here })
.then(() => {
  channel.consume(QUEUE, (message) => {
    console.log('Raw RabbitMQ message received', message)

    // Simulate some job to do
    setTimeout(() => {
      channel.ack(message, false)
    }, 5000})

  }, { noAck: false })
})

// publisher.js
amqp.connect(...)
.then(connection => connection.createConfirmChannel())
.then(() => { assert exchange here })
.then(() => {
  channel.publish(exchange, routingKey, new Buffer(...),{}, (err, ok) => {
    if (err) {
      console.log('Error from handling confirmation on publisher side', err)
    } else {
      console.log('From handling confirmation on publisher side', ok)
    }
  })
})

Running the example, i can see following logs:

From handling confirmation on publisher side undefined
Raw RabbitMQ message received
Time to ack the message

As far as i see, at least by this log, publisher will be notified only when message was enqueued? (So having consumer acking the message will not influence publisher in any way)

Quoting further:

If a message cannot be routed, the broker will send a Basic.Nack RPC request indicating the failure. It is then up to the publisher to decide what to do with the message.

Changing the above example, where i only changed the routing key of the message to something that should not be routed anywhere (there are no bindings that would match routing key), from logs i can see only following.

From handling confirmation on publisher side undefined

Now i'm more confused, about what publisher is notified exactly here? I would understand that it receive an error, like Can't route anywhere, that would be aligned with quote above. But as you can see err is not defined and as side question even if amqplib in their official docs are using (err, ok), in no single case i see those defined. So here output is same like in above example, how one can differ between above example and un-routable message.

So what im up to here, when exactly publisher will be notified about what is happening with the message? Any concrete example in which one would use PublisherConfirms? From logging above, i would conclude that is nice to have it in cases where you want to be 100% sure that message was enqueued.

like image 483
Srle Avatar asked Jan 24 '17 03:01

Srle


People also ask

What publisher confirms RabbitMQ?

Publisher confirms are a RabbitMQ extension to implement reliable publishing. When publisher confirms are enabled on a channel, messages the client publishes are confirmed asynchronously by the broker, meaning they have been taken care of on the server side.

How do you handle failed messages in RabbitMQ?

Acknowledge the original message, and put the new message onto the queue. Then you can check the retries count each time the message goes to a worker for processing. If it fails, either create a new message with updated metadata or permanently fail the message by sending reject with requeue set to false.

Does the publisher know who the message is sent to?

PublisherThe Publisher does not know the recipient, but it gets pinged if the message delivery to the Topic is not successful.

How do I see published messages in RabbitMQ?

In rabbitmq, we can read or consume a published messages from queue using web management portal for that we need to login into rabbitmq web management portal using default (guest) credentials like as shown below.


2 Answers

After searching again and again i have found this http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/

The basic rules are as follows:

  1. An un-routable mandatory or immediate message is confirmed right after the basic.return
  2. transient message is confirmed the moment it is enqueued
  3. Persistent message is confirmed when it is persisted to disk or when it is consumed on every queue.

If more than one of these conditions are met, only the first causes a confirm to be sent. Every published message will be confirmed sooner or later and no message will be confirmed more than once.

like image 100
Srle Avatar answered Nov 15 '22 21:11

Srle


by default publishers don't know anything about consumers.

PublisherConfirms is used to check if the message reached the broker, but not if the message has been enqueued.

you can use mandatory flag to be sure the message has been routed see this https://www.rabbitmq.com/reliability.html

To ensure messages are routed to a single known queue, the producer can just declare a destination queue and publish directly to it. If messages may be routed in more complex ways but the producer still needs to know if they reached at least one queue, it can set the mandatory flag on a basic.publish, ensuring that a basic.return (containing a reply code and some textual explanation) will be sent back to the client if no queues were appropriately bound.

like image 37
Gabriele Santomaggio Avatar answered Nov 15 '22 21:11

Gabriele Santomaggio