Mongo DB: Get all documents inserted after the last known one

Question

> db.events.find()
{ "_id" : ObjectId("50911c3e09913b2c643f1215"), "context" : "jvc8irfjc9cdnf93", "key" : "value" }
{ "_id" : ObjectId("50911c4709913b2c643f1216"), "context" : "jvc8irfjc9cdnf93", "key" : "new value" }
{ "_id" : ObjectId("50911c4b09913b2c643f1217"), "context" : "jvc8irfjc9cdnf93", "key" : "newer value" }
{ "_id" : ObjectId("50911c4f09913b2c643f1218"), "context" : "jvc8irfjc9cdnf93", "key" : "newest value" }
{ "_id" : ObjectId("50911c6309913b2c643f1219"), "context" : "0djd8vcndkfnjhv3", "key" : "value" }

is a list of events on a Mongo DB server. The client keeps track of the last event he knows about, for example 50911c4709913b2c643f1216, the second one.

How to I get all events that have been inserted after this known one? In this case 50911c4b09913b2c643f1217, 50911c4f09913b2c643f1218 and 50911c6309913b2c643f1219.

Answer 1

Since the ObjectId by default contains a inc and a timestamp ( http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-BSONObjectIDSpecification ) you can actually use the ObjectId to understand insertion time (roughly) via:

db.col.find({_id: {$gt: {ObjectId("50911c4709913b2c643f1216")}}});

However the ObjectId cannot always be extremely reliable especially if, in your app, you create the ObjectId a while before insertion and actually one record which you know was created later actually shows (because of its _id) as earlier.

For insertions in certain scenarios where you cannot trust the ObjectId for insertion timer you can add a ts field of a BSON Date type (ISODate) and range on that using two queries, one to get the original documents time and then another to get all documents after.

I personally prefer to just go for the second method of using a BSON date type since it is more reliable and flexible.

Answer 2

The reason why you are seeing the object ids in increasing order here is because the spec says that time|machine|pid|inc is the format for creating the ObjectId.

Notice that there is already a time component in the ObjectId , but that is in seconds.
The Date type in Mongo is the representation of the number of milliseconds from epoch, which will give you some more precision for figuring out time of insertion.

I think the best way is to use a counter in the form of Sequence numbers if you need absolute precision beyond milli seconds, but if you think that your app is not that write intensive just use the Date type in the form of an additional key called timestamp.