Running multiple instances of celery on the same server

Question

I want to run two instances of celery on the same machine. One is for an 'A' version of my application, the other is for the 'B' version.

I have two instances, which I start like this:

(env1)/home/me/firstapp$ celery -A app.tasks worker --config celeryconfig
(env2)/home/me/secondapp$ celery -A app.tasks worker -n Carrot --config celeryconfig

In tasks.py in each application, I create a celery instance like this:

 celery = Celery('tasks', backend='amqp', broker='amqp://[email protected]//')
 @celery.task
 def run_a_task():
     do_stuff()

In env2's task.py, how can I specify that I want to use the second celery instance from secondapp(named Carrot), rather than the first one from firstapp? I suspect I need to change something in the constructor for celery on the first line, but I don't know what to add.

Answer 1

I solved this by using a virtual host for celery.

Once the rabbitmq server is running I issue these commands:

rabbitmqctl add_user user password
rabbitmqctl add_vhost app2
rabbitmqctl set_permissions -p app2 user ".*" ".*" ".*"

Then I start celery with:

celery -A tasks worker --broker=amqp://user:password@localhost/app2

With my task, I initialize the celery object like this:

celery = Celery('tasks', backend='amqp', broker='amqp://user:password@localhost:5672/app2