Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery: When should you choose Redis as a message broker over RabbitMQ?

My rough understanding is that Redis is better if you need the in-memory key-value store feature, however I am not sure how that has anything to do with distributing tasks?

Does that mean we should use Redis as a message broker IF we are already using it for something else?

like image 355
Pig Avatar asked Apr 06 '17 20:04

Pig


People also ask

Is Redis a good message broker?

Redis Pub/Sub is an extremely lightweight messaging protocol designed for broadcasting live notifications within a system. It's ideal for propagating short-lived messages when low latency and huge throughput are critical. Redis Lists and Redis Sorted Sets are the basis for implementing message queues.

Why RabbitMQ is better than Redis?

RabbitMQ vs Redis - Differences Between Message Brokers RabbitMQ is designed as a dedicated message broker, whereas Redis is a database that you can use as a message broker. Mostly, RabbitMQ outperforms Redis and guarantees message delivery with the help of message durability and acknowledgments.

When should I use a message broker?

For example, a message broker may be used to manage a workload queue or message queue for multiple receivers, providing reliable storage, guaranteed message delivery and perhaps transaction management.


1 Answers

I've used both recently (2017-2018), and they are both super stable with Celery 4. So your choice can be based on the details of your hosting setup.

  • If you must use Celery version 2 or version 3, go with RabbitMQ. Otherwise...
  • If you are using Redis for any other reason, go with Redis
  • If you are hosting at AWS, go with Redis so that you can use a managed Redis as service
  • If you hate complicated installs, go with Redis
  • If you already have RabbitMQ installed, stay with RabbitMQ

In the past, I would have recommended RabbitMQ because it was more stable and easier to setup with Celery than Redis, but I don't believe that's true any more.


Update 2019

AWS now has a managed service that is equivalent to RabbitMQ called Amazon MQ, which could reduce the headache of running this as a service in production. Please comment below if you have any experience with this and celery.

like image 100
Mark Chackerian Avatar answered Sep 25 '22 04:09

Mark Chackerian