Multi Celery projects with same RabbitMQ broker backend process

Question

How can I use two different celery project which consumes messages from single RabbitMQ installation.

Generally, these scripts work fine if I use different rabbitmq for them. But on production machine, I need to share the same RabbitMQ backend for them.

Note: Due to some constraint, I cannot merge new projects in existing, so it will be two different project.

Answer 1

RabbitMQ has the ability to create virtual message brokers called virtual hosts or vhosts. Each one is essentially a mini-RabbitMQ server with its own queues. This lets you safely use one RabbitMQ server for multiple applications.

rabbitmqctl add_vhost command creates a vhost.

By default Celery uses the / default vhost:

celery worker --broker=amqp://guest@localhost//

But you can use any custom vhost:

celery worker --broker=amqp://guest@localhost/myvhost

Examples:

rabbitmqctl add_vhost new_host
rabbitmqctl add_vhost /another_host

celery worker --broker=amqp://guest@localhost/new_host

celery worker --broker=amqp://guest@localhost//another_host