Menu
I am running Django with Celery on top of RabbitMQ as a queue to handle some data processing tasks. I am kicking off celery tasks when a user first signs up, as well as periodically to update their data. However, I'd like to of course give priority to the tasks running users who are currently online. I noticed there was a priority setting for tasks in celery, but it seems that rabbitmq does not support this. This thread http://groups.google.com/group/celery-users/browse_thread/thread/ac3b6123d63421e5/b7740def1389e87e?lnk=gst&q=priority#b7740def1389e87e suggests have two different queues, a high priority one and a low priority one, or setting a rate limit for lower priority tasks.
Does anyone have a good workaround to implement priority? Thanks in advance!
456
Sometimes messages simply have a higher priority and need to be handled before other messages. Thankfully, RabbitMQ supports priority queues.
Installation & configuration We will install celery using pip. We don't use sudo as we are installing celery to our virtual environment. However, we also need to install rabbitmq on the system as it runs in the background. The -detached option allows us to run rabbitmq-server in the background.
Clients submit jobs to the message broker, and the queue is listening for job requests using Celery. The message broker then distributes job requests to workers. Workers wait for jobs from Celery and execute the tasks. When finished, the worker sends a result to another queue for the client to process.
As I understand it, and I've not yet done this, Ask covers this here:
http://docs.celeryproject.org/en/master/faq.html#does-celery-support-task-priorities [stable] http://docs.celeryproject.org/en/latest/faq.html#does-celery-support-task-priorities [latest]
85
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
