pymongo + gevent: throw me a banana and just monkey_patch?

Question

Quickie here that needs more domain expertise on pymongo than I have right now:

Are the "right" parts of the pymongo driver written in python for me to call gevent monkey_patch() and successfully alter pymongo's blocking behavior on r/w within gevent "asynchronous" greenlets?

If this will require a little more leg work on gevent and pymongo -- but it is feasible -- I would be more than willing to put in the time as long as i can get a little guidance over irc.

Thanks!

Note: At small scale mongo writes are not a big problem because we are just queuing a write "request" before unblocking. BUT talking to fiorix about his twisted async mongo driver (https://github.com/fiorix/mongo-async-python-driver), even mongo's quick write (requests) can cause problems in asyncronous applications at scale. (And of course, non-blocking reads could cause problems from the start!)

Answer 1

I have used PyMongo with Gevent and here are a few things you need to watch out for:

1. Instantiate only one pymongo.Connection object, preferrably as a global or module-level variable. This is important because Connection has within itself a pool!
2. Monkey patch everything, or at least BOTH socket and threading. Due to the use of thread locals in Connection, patching socket alone is not enough.
3. Remember to call end_request to return the connection to the pool.

The answer to your question is go ahead, PyMongo works just fine with Gevent.

Answer 2

On initial inspection it doesn't appear to do any socket operations in the c code so it should be fine (blocking ops should just block the green thread).