Celery : cannot shutdown celery due to PID changes

Question

I'm using Cerely to manage delayed task on my django project.

I got problem when I tried to shutdown celery as suggested in the manual.

>> ps auxww | grep 'celery worker' | awk '{print $2}' | xargs kill -9
   kill: No such process
>> ps auxww | grep 'celery worker' | awk '{print $2}
   28630
>> ps auxww | grep 'celery worker' | awk '{print $2}
   28633

PID continuosly changes and it makes hard to send killing signal.

How can I solve this problem? Thanks in advance.

---

[ Update ]

django settings.py

import djcelery

    djcelery.setup_loader()
    BROKER_URL = 'amqp://guest:guest@localhost:5672/' # Using RabbitMQ
    CELERYD_MAX_TASKS_PER_CHILD = 1

PID check (After reboot)

>> ps auxww | grep 'celery worker' | awk '{print $2}'
   3243
>> manage.py celery worker --loglevel=info
   celery@{some id value}.... ready
>> ps auxww | grep 'celery worker' | awk '{print $2}'
   3285
   3293
   3296
>> ps auxww | grep 'celery worker' | awk '{print $2}' | xargs kill -9
   kill: No such process
>> ps auxww | grep 'celery worker' | awk '{print $2}'
   3321
>> ps auxww | grep 'celery worker' | awk '{print $2}'
   3324

Question

• At least, one celery worker remains even though rebooted. And its PID changes continuously.
• Celery daemon executes two workers at once. How can I fix it to only one worker ?

Answer 1

It is doing precisely what you asked for with the CELERYD_MAX_TASKS_PER_CHILD setting:

Maximum number of tasks a pool worker process can execute before it’s replaced with a new one.

Apparently you wanted to run one worker process, but that is controlled by a different setting, namely CELERYD_CONCURRENCY.

So, replace

CELERYD_MAX_TASKS_PER_CHILD = 1

with

CELERYD_CONCURRENCY = 1