Mongoose Populate returning null or undefined

Question

I'm sorry if this is a n00b question, I've been searching Google & Stack for hours now and I've got to ask!

I have two schemas, User and Story, shown below. I am trying to reference the User for a Story using the Ref option to Populate in a Query - I've using mySQL before and so wanted to try to replicate a JOIN statement. Whenever I try to use populate I just get the objectID returned, or null (shown below).

Edited 12 Nov to fix hardcoded IDs & add console data

story-schema.js

var mongoose = require('mongoose'),
Schema = mongoose.Schema,
User = require('./user-schema');

var StorySchema = new Schema({  
title: { type: String, required: true },  
author_id: { type: Schema.Types.ObjectId, ref: 'User' },
summary: { type: String, required: true },
rating: { type: String, required: true }
});

module.exports = mongoose.model('Story', StorySchema, 'stories');

user-schema.js

var mongoose = require('mongoose'),
Schema = mongoose.Schema;

var UserSchema = new Schema({
    username: { type: String, required: true, index: { unique: true } },
    is_admin: {type: Boolean, required: true, default: false }
});

save - id hardcoded for example

app.post('/api/new_story', function(req, res){
var story;
story = new Story({
    title: req.body.title,
    author_id: mongoose.Types.ObjectId(req.params._id), 
            /* ex of req.params._id: 527fc8ff241cdb8c09000003*/
    summary: req.body.summary,
    rating: req.body.rating
});

story.save(function(err) {
    if (!err) {
        return console.log("created");
    } else {
        return console.log(err);
    }
});
return res.send(story);
});

example user when logged in terminal

{
"__v" : 0,
"_id" : ObjectId("527fc8ff241cdb8c09000003"),
"is_admin" : false,
"username" : "ted"
}

example story when logged in terminal

{
"title" : "Test Story",
"author_id" : "527fc8ff241cdb8c09000003",
"summary" : "Test summary",
"rating" : "12",
"_id" : ObjectId("52827692496c16070b000002"),
"__v" : 0
}

queries

//other mongoose/app includes above
User = require('./config/schema/user-model'),
Story = require('./config/schema/story-model');

// data.author_id = 527fc8ff241cdb8c09000003 
// data.author_id.username = undefined
app.get('/api/query/:id', function (req, res){
return Story.findOne({_id:req.params.id})
.populate( { path: 'User' } )
.exec(function (err, data) {
            console.log(data.author_id);
            console.log(data.author_id.username);
          if (err) {
            return res.json({error:err})
          }
        })
});

// data.author_id = null 
app.get('/api/query2/:id', function (req, res){
return Story.findOne({_id:req.params.id}) //_id hardcoded for example
.populate( 'author_id' )
.exec(function (err, data) {
            console.log(data.author_id);
          if (err) {
            return res.json({error:err})
          }
        })
});

In the first query I get the author_id I already saved back, which kind of makes sense as that's what I saved - but I access the username.

In the second query I can't even access the author_id I've already saved.

Edit: I can run a normal GET query fine without the 'populate'

What I'd like to happen

Is to be able to access the author information from the story - this is more like a proof of concept. Eventually I'd like to reference the Story _id in the the User model as there can be many Stories to a User, but only one User per Story but thought I'd start here first.

Answer 1

I was stuck with this problem for a week. After trying everything and making sure that I had no circular references, I decided to reset my DB.

After removing all collections, and storing them again, all my relations managed by mongoose's populate() worked.

It is so frustrating because this issue is not documented and I lost 20-30 hours trying to make a stupid collection reference work.

Answer 2

Since your example includes raw Object IDs not even quoted as valid javascript strings, it's hard to understand what is a mistake in your question vs a mistake in your code, but mostly your second query looks like it should work. I suspect you are mixing up your ObjectId values. You are using the same ObjectId for your story and your user and that won't work:

   author_id: mongoose.Types.ObjectId(528149b38da5b85003000002),

That's the same ID you use to lookup the story. So I think the fix is:

1. Make sure you have a valid user record in the users collection
2. Make sure your test story record's story.author_id matches that
3. Make sure you are looking up the story by the story Id and not the author_id.

If you still can't get this working, code the entire thing in a single .js file with both schemas, both models, and some code to create the records then do the queries, and post that.

Answer 3

Is it possible that you forget to include a "ref" field in your Schema? I just encountered this issue and after adding a ref field to my Schema it works immediately without any need to reset the database. Hope this information helps.

I found an example documented here: https://mongoosejs.com/docs/3.0.x/docs/populate.html. Although it is a documentation for version 3.0.x, I found it helpful even for version 5.5.5 which is now I am using.