Nodejs + mongodb : How to query $ref fields?

Question

I'am using MongoDB with a nodejs REST service which exposes my data stored inside. I have a question about how to interrogate my data which uses $ref.

Here is a sample of an Object which contains a reference to another object (detail) in anther collection :

{
    "_id" : ObjectId("5962c7b53b6a02100a000085"),
    "Title" : "test",
    "detail" : {
        "$ref" : "ObjDetail",
        "$id" : ObjectId("5270c7b11f6a02100a000001")
    },
    "foo" : bar
}

Actually, using Node.js and mongodb module, I do the following :

db.collection("Obj").findOne({"_id" : new ObjectID("5962c7b53b6a02100a000085"},
function(err, item) {
    db.collection(item.$ref).findOne({"_id" : item.$id}, function(err,subItem){
        ...
    });
});

In fact I make 2 queries, and get 2 objects. It's a kind of "lazy loading" (not exactly but almost)

My question is simple : is it possible to retrieve the whole object graph in one query ?

Thank you

Answer 1

No, you can't.

To resolve DBRefs, your application must perform additional queries to return the referenced documents. Many drivers have helper methods that form the query for the DBRef automatically. The drivers do not automatically resolve DBRefs into documents.

From the MongoDB docs http://docs.mongodb.org/manual/reference/database-references/.

Answer 2

Is it possible to fetch parent object along with it's $ref using single MongoDB query?

No, it's not possible. Mongo have no inner support for refs, so it up to your application to populate them (see Brett's answer).

But is it possible to fetch parent object with all its ref's with a single node.js command?

Yes, it's possible. You can do it with Mongoose. It has build-in ref's population support. You'll need to change your data model a little bit to make it work, but it's pretty much what you're looking for. Of course, to do so Mongoose will make the same two MongoDB queries that you did.

Answer 3

Answer of Vladimir is not still valid as the db.dereference method was deleted from MongoDB Nodejs API:

https://www.mongodb.com/blog/post/introducing-nodejs-mongodb-20-driver

The db instance object has been simplified. We've removed the following methods:

db.dereference due to db references being deprecated in the server

Answer 4

No, very few drivers for MongoDb include special support for a DBRef. There are two reasons:

1. MongoDb doesn't have any special commands to make retrieval of referenced documents possible. So, drivers that do add support are artificially populating the resulting objects.
2. The more, "bare metal" the API, the less it makes sense. In fact, as. MongoDb collections are schema-less, if the NodeJs driver brought back the primary document with all references realized, if the code then saved the document without breaking the references, it would result in an embedded subdocument. Of course, that would be a mess.

Unless your field values vary, I wouldn't bother with a DBRef type and would instead just store the ObjectId directly. As you can see, a DBRef really offers no benefit except to require lots of duplicate disk space for each reference, as a richer object must stored along with its type information. Either way, you should consider the potentially unnecessary overhead of storing a string containing the referenced collection's documents.

Many developers and MongoDb, Inc. have added an object document mapping layer on top of the existing base drivers. One popular option for MongoDb and Nodejs is Mongoose. As the MongoDb server has no real awareness of referenced documents, the responsibility of the references moves to the client. As it's more common to consistently reference a particular collection from a given document, Mongoose makes it possible to define the reference as a Schema. Mongoose is not schema-less.

If you accept having and using a Schema is useful, then Mongoose is definitely worth looking at. It can efficiently fetch a batch of related documents (from a single collection) from a set of documents. It always is using the native driver, but it generally does operations extremely efficiently and takes some of the drudgery out of more complex application architectures.

I would strongly suggest you have a look at the populate method (here) to see what it's capable of doing.

Demo     /* Demo would be a Mongoose Model that you've defined */
.findById(theObjectId)
.populate('detail')
.exec(function (err, doc) {
  if (err) return handleError(err);
  // do something with the single doc that was returned
})

If instead of findById, which always returns a single document, find were used, with populate, all returned documents' details property will be populated automatically. It's smart too that it would request the same referenced documents multiple times.

If you don't use Mongoose, I'd suggest you consider a caching layer to avoid doing client side reference joins when possible and use the $in query operator to batch as much as possible.