Running Plone subscriber events asynchronously

Question

In using Plone 4, I have successfully created a subscriber event to do extra processing when a custom content type is saved. This I accomplished by using the Products.Archetypes.interfaces.IObjectInitializedEvent interface.

configure.zcml

<subscriber 
        for="mycustom.product.interfaces.IRepositoryItem
             Products.Archetypes.interfaces.IObjectInitializedEvent"
        handler=".subscribers.notifyCreatedRepositoryItem"
   /> 

subscribers.py

def notifyCreatedRepositoryItem(repositoryitem, event):
    """
    This gets called on IObjectInitializedEvent - which occurs when a new object is created.
    """ 
    my custom processing goes here. Should be asynchronous

However, the extra processing can sometimes take too long, and I was wondering if there is a way to run it in the background i.e. asynchronously.

Is it possible to run subscriber events asynchronously for example when one is saving an object?

Answer 1

Not out of the box. You'd need to add asynch support to your environment.

Take a look at plone.app.async; you'll need a ZEO environment and at least one extra instance. The latter will run async jobs you push into the queue from your site.

You can then define methods to be executed asynchronously and push tasks into the queue to execute such a method asynchronously.

Example code, push a task into the queue:

from plone.app.async.interfaces import IAsyncService

async = getUtility(IAsyncService)
async.queueJob(an_async_task, someobject, arg1_value, arg2_value)

and the task itself:

def an_async_task(someobject, arg1, arg2):
    # do something with someobject

where someobject is a persistent object in your ZODB. The IAsyncService.queueJob takes at least a function and a context object, but you can add as many further arguments as you need to execute your task. The arguments must be pickleable.

The task will then be executed by an async worker instance when it can, outside of the context of the current request.

Answer 2

Just to give more options, you could try collective.taskqueue for that, really simple and really powerful (and avoid some of the drawbacks of plone.app.async).

The description on PyPI already has enough to get you up to speed in no time, and you can use redis for the queue management which is a big plus.