MongoDB performance of db.eval() in javascript shell vs. pymongo

Question

I need to perform update operations on documents in which I need to rotate some values in an array. MongoDB update queries currently do not allow you to $pop and then $push on the same field in an update. After searching for advice online, I decided that db.eval() would be most appropriate for my use since it ensures atomicity and the operation that I'm performing is very short so it wont lock up the db for too long.

Here's an example of something I'm trying to do:

db.eval(function (id, newVal) {
    doc = db.collection.findOne({_id: id});
    doc.values.shift();
    doc.values.push(newVal);
    db.collection.save(doc);
}, id, newVal);

And this works perfectly! I then enabled mongoDB profiling to see how many milliseconds the eval() command was taking and I'd always get results of less that 1 millisecond:

 > db.system.profile.find({op: "command"}, {"millis": 1}) 
 { "millis" : 0 }
 { "millis" : 0 }
 ...

This is good news to me, except that my application is in python, so I'm using a pymongo client to perform the eval() commands. (The data above is from the mongo shell) But now, when I run identical eval() commands using pymongo:

conn = pymongo.Connection(mongo_server_hostname)
db = conn.my_db

db.eval("""function (id, newVal) {
    doc = db.collection.findOne({_id: id});
    doc.values.shift();
    doc.values.push(newVal);
    db.collection.save(doc);
}""", id, new_val)

I get very different profiling results:

> db.system.profile.find({op: "command"}, {"millis": 1}) 
{ "millis" : 13 }
{ "millis" : 14 }
{ "millis" : 14 }
...

Is there something fundamentally different about running the same eval() commands from within the mongo shell and pymongo that results in the server taking 14ms more to run identical commands from pymongo?

Answer 1

One possible cause (but not necessarily the cause) for the discrepancy that you are seeing is that both the mongo shell and the mongod server use Google's v8 Javascript engine by default (though it can be configured to use Spidermonkey as an alternative) for interpreting the commands that you give.

Google's v8 sees hotspots in Javascript code and is likely to JIT code that is frequently used.

On the other hand, vanilla PyMongo is written in pure Python, which means that it will always be interpreted, which is something that has a considerable overhead.

If you are not doing so already, one possibility would be to use the PyMongo extension written in C instead of default one or, if the rest of your application is compatible, use the PyPy JIT interpreter for Python.

If you use any distribution derived from Debian (like Ubuntu), the package python-pymongo-ext provides you the precompiled version of the C version of PyMongo.