celery memory management

Question

I have a 3 celery nodes running between 6 and 8 workers each and my tasks tend to use a lot of memory. It seems that when the tasks finish, they aren't releasing the memory. Is there a way to either tell the worker process to die and restart after execution completes, or improve celery's garbage collection?

Answer 1

For those workers you could set max-tasks-per-child-setting to one.