Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Appropriate approach for Message Queue / Scheduled tasks in Django

I'm wondering what criteria would need to be considered when we need to use some kind of task queue in a django project, I'm thinking in performace, development speed, flexibility, etc.

I've been using Celery+RabbitMQ and Django-ztask+ZeroMQ indistinctly for a while (I'm sure there are another good ones), but I haven't an accurate canon for picking up the most suitable in each case.

Could you provide some peculiarities for each of them that allows the user chooses between them?, does it might include some another stable MQ approaches as well?

like image 355
trinchet Avatar asked Jul 02 '14 15:07

trinchet


People also ask

Why Celery is used in Django?

Celery makes it easier to implement the task queues for many workers in a Django application.

Can I use Celery without Django?

Yes you can. Celery is a generic asynchronous task queue.

What is Celery used for Python?

Celery is a task queue implementation for Python web applications used to asynchronously execute work outside the HTTP request-response cycle. Celery is an implementation of the task queue concept.

How do I run a Django task?

In Django Background Task, all tasks are implemented as functions (or any other callable). There are two parts to using background tasks: creating the task functions and registering them with the scheduler. setup a cron task (or long running process) to execute the tasks.


1 Answers

I can't provide much but I have used two different solutions, Celery+Redis and Celery+RabbitMQ.

I tried RabbitMQ first and after getting all its dependencies installed and spending some time wading through configs I got it working. It worked well, didn't drop anything, but I was always nervous about restarting (either it or the server) because I was never completely sure it would come back up. I'm sure that's my fault, but I couldn't work out what I'd done wrong.

So I thought I'd give Redis a try. Installed and configured it in about 3 minutes and it has worked without any of my attention since.

Now if only there was something easier to configure than Celery...

like image 153
Daniel Avatar answered Oct 20 '22 21:10

Daniel