Celery. Decrease number of processes

Question

Is there any way around to limit number of workers in celery? I have small server and celery always creates 10 processes on 1 core processor. I want to limit this number to 3 processes.

Answer 1

I tried setting concurrency to 1 and max_tasks_per_child to 1 in my settings.py file and ran 3 tasks at the same time. It just spawns 1 process as a User and the other 2 as celery. It should should just run 1 process and then wait for it to finish before running the other one.

I am using django celery.

EDIT {

I was assigning concurrency by writing CELERYD_CONCURRENCY = 1 in settings.py file. But when I looked at the celery log file using "tail -f /var/log/celery/w1.log" then I saw a value of 8 assigned to concurrency. This told me that setting.py does not change the concurrency. To fix this issue I added the following lines to "/etc/default/celeryd" file.

# Extra arguments to celeryd
CELERYD_OPTS="--concurrency=1"

Now the second task in the queue waits until the first is finished.

}