Python Twisted and database connections

Question

Our projects at work include synchronous applications (short lived) and asynchronous Twisted applications (long lived). We're re-factoring our database and are going to build an API module to decouple all of the SQL in that module. I'd like to create that API so both synchronous and asynchronous applications can use it. For the synchronous applications I'd like calls to the database API to just return data (blocking) just like using MySQLdb, but for the asynchronous applications I'd like calls to the same API functions/methods to be non-blocking, probably returning a deferred. Anyone have any hints, suggestions or help they might offer me to do this? Thanks in advance, Doug

Answer 1

twisted.enterprise.adbapi seems the way to go -- do you think it fails to match your requirements, and if so, can you please explain why?

Answer 2

Within Twisted, you basically want a wrapper around a function which returns a Deferred (such as the Twisted DB layer), waits for it's results, and returns them. However, you can't busy-wait, since that's using up your reactor cycles, and checking for a task to complete using the Twisted non-blocking wait is probably inefficient.

Will inlineCallbacks or deferredGenerator solve your problem? They require a modern Twisted. See the twistedmatrix docs.

def thingummy():
   thing = yield makeSomeRequestResultingInDeferred()
   print thing #the result! hoorj!
thingummy = inlineCallbacks(thingummy)

Another option would be to have two methods which execute the same SQL template, one which uses runInteraction, which blocks, and one which uses runQuery, which returns a Deferred, but that would involve more code paths which do the same thing.