Mongoose - RangeError: Maximum Call Stack Size Exceeded

Question

I am trying to bulk insert documents into MongoDB (so bypassing Mongoose and using the native driver instead as Mongoose doesn't support bulk insert of an array of documents). The reason I'm doing this is to improve the speed of writing.

I am receiving the error "RangeError: Maximum Call Stack Size Exceeded" at console.log(err) in the code below:

function _fillResponses(globalSurvey, optionsToSelectRegular, optionsToSelectPiped, responseIds, callback) {   Response.find({'_id': {$in: responseIds}}).exec(function(err, responses) {     if (err) { return callback(err); }      if (globalSurvey.questions.length) {       responses.forEach(function(response) {         console.log("Filling response: " + response._id);         response.answers = [];         globalAnswers = {};         globalSurvey.questions.forEach(function(question) {           ans = _getAnswer(question, optionsToSelectRegular, optionsToSelectPiped, response);           globalAnswers[question._id] = ans;           response.answers.push(ans);         });       });       Response.collection.insert(responses, function(err, responsesResult) {         console.log(err);         callback()       });     } else {         callback();       }   }); }  

So similar to: https://stackoverflow.com/questions/24356859/mongoose-maximum-call-stack-size-exceeded

Perhaps it's something about the format of the responses array that Mongoose returns that means I can't directly insert using MongoDB natively? I've tried .toJSON() on each response but no luck.

I still get the error even with a very small amount of data but looping through and calling the Mongoose save on each document individually works fine.

EDIT: I think it is related to this issue: http://howtosjava.blogspot.com.au/2012/05/nodejs-mongoose-rangeerror-maximum-call.html

My schema for responses is:

var ResponseSchema = new Schema({   user: {     type: Schema.ObjectId,     ref: 'User'   },   randomUUID: String,   status: String,   submitted: Date,   initialEmailId: String,   survey: String,   answers: [AnswerSchema] });  

So, answers are a sub-document within responses. Not sure how to fix it though....

Answer 1

I was having this same issue and I started digging through the mongoose source code (version 3.8.14). Eventually it led me to this line within

• mongoose/node_modules/mongodb/lib/mongodb/collection/core.js -> insert(...) -> insertWithWriteCommands(...) ->

• mongoose/node_modules/mongodb/lib/mongodb/collection/batch/ordered.js -> bulk.insert(docs[i]) -> addToOperationsList(...) -> bson.calculateObjectSize(document, false);

  var bsonSize = bson.calculateObjectSize(document, false);

Apparently, this calls BSON.calculateObjectSize, which calls calculateObjectSize which then infinitely recurses. I wasn't able to dig that far in to what caused it, but figured that it may have something to do with the mongoose wrapper binding functions to the Schema. Since I was inserting raw data into mongoDB, once I decided to change the bulk insert in mongoose to a standard javascript object, the problem went away and bulk inserts happened correctly. You might be able to do something similar.

Essentially, my code went from

//EDIT: mongoose.model needs lowercase 'm' for getter method  var myModel = mongoose.model('MyCollection'); var toInsert = myModel(); var array = [toInsert]; myModel.collection.insert(array, {}, function(err, docs) {}); 

to

//EDIT: mongoose.model needs lowercase 'm' for getter method  var myModel = mongoose.model('MyCollection'); var toInsert = { //stuff in here     name: 'john',    date: new Date() }; var array = [toInsert]; myModel.collection.insert(array, {}, function(err, docs) {}); 

Answer 2

Confirmed, but not a bug. Model.collection.insert() bypasses Mongoose and so you're telling the node driver to insert an object that contains mongoose internals like $__, etc. The stack overflow is probably because bson is trying to compute the size of an object that references itself indirectly.

Long story short, use Document.toObject(), that's what its for: http://mongoosejs.com/docs/api.html#document_Document-toObject

Response.find({}).exec(function(err, responses) {   if (err) {      return callback(err);    }    if (true) {     var toInsert = [];     responses.forEach(function(response) {       console.log("Filling response: " + response._id);       response.answers = [];       [{ name: 'test' }].forEach(function(ans) {         response.answers.push(ans);       });       toInsert.push(response.toObject());     });     Response.collection.insert(toInsert, function(err, responsesResult) {       console.log(err);     });   } else {       callback();     } }); 

Also, the code you specified won't work even if you fix the stack overflow. Since you're trying to insert() docs that are already in the database, all the inserts will fail because of _id conflicts. You'd really be much better off just using a stream() to read the results one at a time and then save() them back into the db.