Can you run Celery in a different container from Django?

Question

From my reading today, in all the examples I found I didn't see any where celery is in a completely separate container from Django itself. It seems as though Celery has to be in the same container since it walks the apps source files and looks for tasks.py as well as the initial celery.py Is that correct or did I misread today?

For example. I am familiar with using docker-compose to spin up Django, Nginx, Postgres and a storage container. I assumed I'b be adding a celery and rabbitmq container, but I see no way to configure Django to use a remote Celery server.

I'm still early in my understanding of Celery, I hope this isn't something I overlooked elsewhere.

Thanks,
-p

Answer 1

By default, that's what happen if you use Heroku, it run a web Dyno for django to respond to requests, and an other worker Dyno for Celery, each Dyno run on a separate instance.

Both Dynos run the same code, your celery worker need to access the models, and it's easy to manage/deploy one code base, but there is nothing stopping you from using different code base for each instance, as the communication between Django and Celery is done with AMQP protocol throw a Broker like Reddis.