How to purge tasks in celery queues using Redis as the broker

Question

Part1

I've read and tried various SO threads to purge the celery tasks using Redis, but none of them worked. Please let me know how to purge tasks in celery using Redis as the broker.

Part 2

Also, I've multiple queues. I can run it within the project directory, but when demonizing, the workers dont take task. I still need to start the celery workers manually. How can I demozize it?

Here is my celerd conf.

# Name of nodes to start, here we have a single node
CELERYD_NODES="w1 w2 w3 w4"


CELERY_BIN="/usr/local/bin/celery"

# Where to chdir at start.
CELERYD_CHDIR="/var/www/fractal/parser-quicklook/"

# Python interpreter from environment, if using virtualenv
#ENV_PYTHON="/somewhere/.virtualenvs/MyProject/bin/python"

# How to call "manage.py celeryd_multi"
#CELERYD_MULTI="/usr/local/bin/celeryd-multi"

# How to call "manage.py celeryctl"
#CELERYCTL="/usr/local/bin/celeryctl"

#CELERYBEAT="/usr/local/bin/celerybeat"

# Extra arguments to celeryd
CELERYD_OPTS="--time-limit=300 --concurrency=8  -Q BBC,BGR,FASTCOMPANY,Firstpost,Guardian,IBNLIVE,LIVEMINT,Mashable,NDTV,Pandodaily,Reuters,TNW,TheHindu,ZEENEWS "

# Name of the celery config module, don't change this.
CELERY_CONFIG_MODULE="celeryconfig"

# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
#CELERYD_USER="nobody"
#CELERYD_GROUP="nobody"

# Set any other env vars here too!
PROJET_ENV="PRODUCTION"

# Name of the projects settings module.
# in this case is just settings and not the full path because it will change the dir to
# the project folder first.
CELERY_CREATE_DIRS=1

Celeryconfig is already provided in part1.

Here is my proj directory structure.

project
|-- main.py
|-- project
|   |-- celeryconfig.py
|   |-- __init__.py
|-- tasks.py

How can I demonize with the Queues? I have provided the queues in CELERYD_OPTS as well.

Is there a way in which we can dynamically demonize the number of queues in the celery? For eg:- we have CELERY_CREATE_MISSING_QUEUES = True for creating the missing queues. Is there something similar to daemonize the celery queues?

Answer 1

If you have several queues, celery purge will purge the default one. You can specify which queue(s) you would like to purge as such:

celery purge -A proj -Q queue1,queue2

Answer 2

celery purge should be enough to clean up the queue in redis. However, your worker will have its own reserved tasks and it will send them back to the queue when you stop the worker. So, first, stop all the workers. Then run celery purge.