django-celery: Dynamically create and register a task

Question

I'd like to add delaying of arbitrary tasks using django-celery. Currently, I have created a class similar to the one below (just an example, actual classes have more than this):

from celery.task import task

class Delayer(object):
    def delay(self, func, minutes):
        return task(func, name="%s.delayed"%self.__class__.__name__)\
            .apply_async(countdown=minutes*60)

I'm running celeryd as follows:

python manage.py celeryd -E -B -lDEBUG

When I try running my delay method from within a django shell [eg Delayer().delay(lambda: 1, 1)], I'm getting an error like this in my celeryd output:

[2013-01-02 15:26:39,324: ERROR/MainProcess] Received unregistered task of type "Delayer.delayed".
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you are using relative imports?
Please see http://bit.ly/gLye1c for more information.

The full contents of the message body was:
{'retries': 0, 'task': "Delayer.delayed", 'eta': '2013-01-02T21:27:39.320913', 'args': [], 'expires': None, 'callbacks': None, 'errbacks': None, 'kwargs': {}, 'id': '99d49fa7-bd4b-40b0-80dc-57309a6f19b1', 'utc': True} (229b)

Traceback (most recent call last):
  File "/home/simon/websites/envs/delayer/local/lib/python2.7/site-packages/celery/worker/consumer.py", line 432, in on_task_received
    strategies[name](message, body, message.ack_log_error)
KeyError: "Delayer.delayed"

My question is, is it possible to register such dynamically created tasks? If not, what other method can I use to achieve the same effect using celery?

Answer 1

The simple answer is that you can't; because celery is running in a different process, it needs to be able to import any code that is run as a celery task; your generated callable is not, so celery's way of moving around references to callables doesn't work.

However this does suggest a possible way of attacking things: if you can come up with a different way of serializing your callable, then you can provide it as an argument to a simple celery task. This previous question may help. Note the cautionary mentions of security :-)