How to handle Mongoose DB connection interruptions

Question

I've been evaluating Mongoose (an ORM for node.js which uses MongoDB for persistent storage).

What I'd like to do is make sure that the app can run when the DB is not up when the app starts, and also handles the DB going down intelligently.

Currently my test app which does not work in either case does this:

var mongoose_connection = mongoose.createConnection(DATABASE_URL, {server:{poolSize:4}});

Then I use that connection when making Models.

In the case when the DB is down at the start of the app then any save() calls on instances silently fail with no error. If the DB comes back up they never get written.

So I would need to detect that the connection never happened and have the app be able to tell that at runtime so that I can handle it somehow.

When the DB goes down after the app has started though the save() calls still do not cause errors but they are queued and the written when the DB comes back.

That seems fine except I'd like to hook into the API to get events when the DB is down and to query how many save's are queued. At some point I might have so many queued events that I would want to just stop making new ones and have the app back off.

Answer 1

Case #1: db is down on app startup. there is a minor bug preventing this use case that I'm fixing now. However, here is the work-around:

var db = mongoose.createConnection();
db.on('error', function (err) {
  if (err) // couldn't connect

  // hack the driver to allow re-opening after initial network error
  db.db.close();

  // retry if desired
  connect();
});

function connect () {
  db.open('localhost', 'dbname');
}

connect();

https://gist.github.com/2878607

An ugly but working gist. First shutdown mongo, then run this gist.

Notice the connection failures.

Then start mongo and see all of the queued up inserts complete and dumped to the console. Writes and finds will begin.

Shutdown mongo, notice that the inserts and finds are being attempted but no callbacks are executed.

Restart mongo. Notice all of the queued inserts and finds are completed.