CeleryBeat Process consumes all OS memory

Question

We are using

django-celery==3.1.10
celery==3.1.20
python 2.7.13

We have written a CustomDataBaseScheduler to schedule task, which schedules the task perfectly on time. We are running CeleryBeat Process as init script, but celeryBeat consumes Full memory of the system i.e. 24GB in a day.

I tried to run pmap on celerybeat Process, but it shows [anon] has took the most memory.

Can someone please help to debug and fix this.

Answer 1

First of all if you are on django 1.8 or above please use celery 4.0 and above. In that case you will not require django-celery. Also in that case follow this tutorial. http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html

Coming to your problem it may be one of the following reasons:

1. Your workers are overloaded. Try using concurrency as mentioned here
2. Check if your django settings DEBUG is set to True. It can cause memory leakages and celery advises against it when you run it.
3. Check for memory leak fixes in the history. To me it seemed the next version after yours, 3.1.21 to be precise has some memory leak fixes. Try upgrading to the latest 3.x version(only if you cannot use 4.x for any reason.)
4. If all else fails, try some monitoring tools to debug the scenario about what is happening. Some monitoring tools are mentioned here.