I am very new to RabbitMQ.
I have set up a 'topic' exchange. The consumers may be started after the publisher. I'd like the consumers to be able to receive messages that have been sent before they were up, and that was not consumed yet.
The exchange is set up with the following parameters:
exchange_type => 'topic' durable => 1 auto_delete => 0 passive => 0
The messages are published with this parameter:
delivery_mode => 2
Consumers use get() to retrieve the messages from the exchange.
Unfortunately, any message published before any client was up is lost. I have used different combinations.
I guess my problem is that the exchange does not hold messages. Maybe I need to have a queue between the publisher and the consumer. But this does not seem to work with a 'topic' exchange where messages are routed by a key.
How should I proceed? I use the Perl
binding Net::RabbitMQ
(shouldn't matter) and RabbitMQ 2.2.0
.
RabbitMQ has great durability support, but it's usually not enabled by default in the drivers. To persist messages to disk, and thus survive a server restart, you need to publish to a durable exchange, the receiving queue has to be durable, and you got to set the "persistent" flag on the message you're publishing.
Topic RabbitMQ exchange type sends messages to queues depending on wildcard matches between the routing key and the queue binding's routing pattern. Messages are routed to one or more queues based on a pattern that matches a message routing key. A list of words separated by a period must be used as the routing key (.).
Topic exchange is a built-in exchange in RabbitMQ, enabling the use of wildcards in the binding key. In RabbitMQ, messages are published to an exchange and, depending on the type of exchange, the message gets routed to one or more queues. RabbitMQ has some built-in exchanges and some that are enabled via a plugin.
In RabbitMQ, there are four different types of exchanges that route the message differently using different parameters and bindings setups. Clients can create their own exchanges or use the predefined default exchanges which are created when the server starts for the first time.
You need a durable queue to store messages if there are no connected consumers available to process the messages at the time they are published.
An exchange doesn't store messages, but a queue can. The confusing part is that exchanges can be marked as "durable" but all that really means is that the exchange itself will still be there if you restart your broker, but it does not mean that any messages sent to that exchange are automatically persisted.
Given that, here are two options:
I would go for #1. There may not be many steps to perform and you could always script the steps required so that they could be repeated. Plus if all your consumers are going to pull from the same single queue (rather than have a dedicated queue each) it's really a minimal piece of administrative overhead.
Queues are something to be managed and controlled properly. Otherwise you could end up with rogue consumers declaring durable queues, using them for a few minutes but never again. Soon after you'll have a permanently-growing queue with nothing reducing its size, and an impending broker apocalypse.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With