mongoDB getLastError() with javascript and node.js not working

Question

I've got a node.js application using mongoDB and I have a function that is used to drop all documents in a collection and then repopulate the collections in my db with some sample documents.

I use this to test my app with known data once in a while when I write new features.

The problem I run into is that if I call drop() on all my collections then call some inserts to repopulate, sometimes i will have less records than I am expecting to be inserted in the database and sometimes whole collections will be missing. my count checks all say # records inserted which matches the expected amount but an entire collection could be missing even though it says its been inserted.

I can only assume since it only happens about 50% of the time that perhaps somehow the inserts start inserting before the drop() completes and then the drop() wipes out the inserted documents because its running async.

I use {safe:true} on all my insert/update commands but db.collection.drop() doesn't take any arguments and there seems to be no way to specify a safe option.

so I am trying to use db.getLastError() after a drop() to block the inserts until the drop completes.

using db.getLastError() or db.getlasterror() throws an error: TypeError: Object #<Db> has no method 'getlasterror'

simplified call to getLastError:

var mongo = require("mongodb");

  var db_conn = new mongo.Db(dbName, new mongo.Server(host, port, { auto_reconnect: true }), {});
  db_conn.open(function(err, db) {
    if(!err) {
      console.log("Database connection: \033[32mopen\033[0m");
    } else {
      console.log("Database connection: \033[31mfailed\033[0m");
    }
  });
  db_conn.getLastError();
  this.db_conn = db_conn;
  this.users = db_conn.collection("users");
  this.companies = db_conn.collection("companies");
  this.suppliers = db_conn.collection("suppliers");

I've tried this on the mongodb module version 1.1.4 and 1.1.7 and just get an error.

Am I using a wrong reference? I can't find anything on stackoverflow or the web in general on getlasterror in a code sample in JavaScript.

If there is a better way to do this I'm all ears.

js docs: http://docs.mongodb.org/manual/reference/javascript/#db.getLastError

mongo docs: http://www.mongodb.org/display/DOCS/getLastError+Command

---- UPDATE Sept 22 2012

Using the answer below I was able to debug a bit what the root problem was and it's as I expected. With the default connection pool of 5 open connections the drops get fired off randomly and the inserts then follow. Depending on the order they get sent in, and the pools they each go to, if one drop finishes fast and then does an insert quickly before the drop for that collection has finished, it would remove the inserted records later.

My fix was to wrap up all the drops in a chaining sequence within the callback of a drop() command so each drop happens after the callback for the previous drop is fired and then at the end call a callback that fires off all the inserts. It's messy, but I've tested the initializedb function now about 25 times with a basic record set of 20 simple inserts and with a loadtest of 100,000 records. no fails yet.

Time will tell but this .lastError() answer worked to solve this problem.

---- UPDATE Sept 30 2012

I wanted to test this for a week or so before responding back to make sure It actually worked. I've run my initialization script about 100 times in the last week both on a base set of 26 records and with sets of 100,000 records all being inserted in bulk. I haven't seen it fail once now using my new code.

Code I use for dropping the collections: (I have a DatabaseHandler object I use with prototype functions for operations)

DatabaseHandler.prototype.dropAll = function(callback, callbackArg) {
  // drop all the documents
  var that = this;
  var finished = callback || function() { };
  var finishedArg = callbackArg;
  this.companies.drop(function(err, reply) {
    if(reply) {
      console.log("[db] Companies collection dropped");
    } else {
      console.log("\033[31m[Error]\033[0m Companies collection failed to drop");
    }
    // drop all the user documents
    that.users.drop(function(err, reply) {
      if(reply) {
        console.log("[db] Users collection dropped");
      } else {
        console.log("\033[31m[Error]\033[0m Users collection failed to drop");
      }
      // drop all the course documents
      that.courses.drop(function(err, reply) {
        if(reply) {
          console.log("[db] Courses collection dropped");
        } else {
          console.log("\033[31m[Error]\033[0m Courses collection failed to drop");
        }
        // drop all the course purchase documents
        that.course_purchases.drop(function(err, reply) {
          if(reply) {
            console.log("[db] Course purchases collection dropped");
          } else {
            console.log("\033[31m[Error]\033[0m Course purchases collection failed to drop");
          }
          console.log("Dropped all documents and collections");
          return finished(finishedArg);
        });
      });
    });
  });
};

Then i have an initialize function that takes an argument to specify if i want extra load test data initialize(addloadtestdata). if i specify dropdata it calls dropAll and takes the initialize function for a callback. so when the drop is finished it calls the initialize callback and starts re-adding documents only after its finished dropping. I also have a small count function that checks how many documents are inserted after finishing the insert as well to verify the correct number were inserted and none were missed. this was probably the best feature i added of all so every time i initialize i can see if the exact number of documents inserted was correct or if i was missing even 1.

function initialize(doLoadtest){
  // do work here
}
// drop all collections and documents
if(dropdata) {
  // drop all the data first then call the initialize as a callback function when completed dropping
  Database.dropAll(initialize, includeLoadtestData);
} else {
  initialize(includeLoadtestData);
}

hope that helps

Answer 1

Are you using the node-mongodb-native driver? If so, take a look at the lastError command.

http://mongodb.github.com/node-mongodb-native/api-generated/db.html#lasterror