Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery is refusing to deserialize content of my custom serialization throwing ContentDisallowed Exception

I have been using celery with rabbitmq as my backend queue to manage my tasks. There are objects containing datetime attributes which cannot be serialized using json and so I chose to use pickle as serializer.

Problem being security related it reports untrusted content cannot be deserialzed. After going through this link http://celery.readthedocs.org/en/latest/userguide/security.html#guide-security I have generated private key and associated self-signed certificate usinf openssl and configured auth settings accordingly. This time I see the error as ExpiredCerticate for which I didn't find any related docs.

So I wrote custom seriailizer as following after which It throws same kind of error as pickle saying

import json
from datetime import datetime
from time import mktime

class DateTimeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return {
                '__type__': '__datetime__',
                'epoch': int(mktime(obj.timetuple()))
            }
        else:
            return json.JSONEncoder.default(self, obj)

def datetime_decoder(obj):
    if '__type__' in obj:
        if obj['__type__'] == '__datetime__':
            return datetime.fromtimestamp(obj['epoch'])
    return obj

# Encoder function                                                                                                                                                                                               
def datetime_dumps(obj):
    return json.dumps(obj, cls=DateTimeEncoder)

# Decoder function                                                                                                                                                                                               
def datetime_loads(obj):
    return json.loads(obj, object_hook=datetime_decoder)

ContentDisallowed: Refusing to deserialize untrusted content of type serializer (application/x-serializer)

Can someone please guide how do I proceed. Any help is appreciated.

Thanks in Advance!

like image 557
Satish Reddy Avatar asked Dec 20 '14 11:12

Satish Reddy


1 Answers

There is not exactly a solution for the above , but you can the following work around. By default in kombu serialization file (serialization.py) invokes a function that disables all the available serializers (pickle included)

from kombu.serialization import registry
registry.enable('pickle')

It will enable pickle serializer disregarding whether content is trusted or untrusted. ofcourse thi is a hack . Hope it helps you!

like image 199
Bharath Potla Avatar answered Oct 31 '22 20:10

Bharath Potla