Celery will refuse to accept pickle by default, should I disable it?

Question

I just setup my first instance of Celery and I am getting warnings to disable pickle as a serializer for security concerns. What is the consequence of turning it off? What is it that Celery needs the serialzer for?

If you depend on pickle then you should set a setting to disable this warning and to be sure that everything will continue working when you upgrade to Celery 3.2::

CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']

Answer 1

From http://celery.readthedocs.org/en/latest/faq.html#is-celery-dependent-on-pickle:

The default serialization format is pickle simply because it is convenient (it supports sending complex Python objects as task arguments).

Whether you will use pickle or not you may want to turn off this warning by setting the CELERY_ACCEPT_CONTENT configuration variable.

In case you don't use Python objects as arguments in your tasks you should consider using safe e.g. json serialization format.

Another limitation of pickle is no task results - see http://docs.celeryproject.org/en/latest/faq.html#isn-t-using-pickle-a-security-concern:

For the task messages you can set the CELERY_TASK_SERIALIZER setting to “json” or “yaml” instead of pickle. There is currently no alternative solution for task results (but writing a custom result backend using JSON is a simple task)