When to close MongoDB database connection in Nodejs

Question

Working with Nodejs and MongoDB through Node MongoDB native driver. Need to retrieve some documents, and make modification, then save them right back. This is an example:

Click to copy

db.open(function (err, db) {
  db.collection('foo', function (err, collection) {
    var cursor = collection.find({});
    cursor.each(function (err, doc) {
      if (doc != null) {
        doc.newkey = 'foo'; // Make some changes
        db.save(doc); // Update the document
      } else {
        db.close(); // Closing the connection
      }
    });
  });
});

With asynchronous nature, if the process of updating the document takes longer, then when cursor reaches the end of documents, database connection is closed. Not all updates are saved to the database.

If the db.close() is omitted, all the documents are correctly updated, but the application hangs, never exits.

I saw a post suggesting using a counter to track number of updates, when fall back to zero, then close the db. But am I doing anything wrong here? What is the best way to handle this kind of situation? Does db.close() have to be used to free up resource? Or does a new db connection needs to open?

Answer 1

Here's a potential solution based on the counting approach (I haven't tested it and there's no error trapping, but it should convey the idea).

The basic strategy is: Acquire the count of how many records need to be updated, save each record asynchronously and a callback on success, which will decrement the count and close the DB if the count reaches 0 (when the last update finishes). By using {safe:true} we can ensure that each update is successful.

The mongo server will use one thread per connection, so it's good to either a) close unused connections, or b) pool/reuse them.

Click to copy

db.open(function (err, db) {   db.collection('foo', function (err, collection) {     var cursor = collection.find({});     cursor.count(function(err,count)){       var savesPending = count;        if(count == 0){         db.close();         return;       }        var saveFinished = function(){         savesPending--;         if(savesPending == 0){           db.close();         }       }        cursor.each(function (err, doc) {         if (doc != null) {           doc.newkey = 'foo'; // Make some changes           db.save(doc, {safe:true}, saveFinished);         }       });     })   }); }); 

Answer 2

It's best to use a pooled connection and then call db.close() in cleanup function at the end of your application's life:

Click to copy

process.on('SIGINT', cleanup);
process.on('SIGTERM', cleanup);

See http://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html

A bit old thread, but anyway.

Answer 3

I found that using counter may apply to simple scenario, but may be hard in complicated situations. Here is a solution that I come up by closing the database connection when database connection is idle:

Click to copy

var dbQueryCounter = 0;
var maxDbIdleTime = 5000; //maximum db idle time

var closeIdleDb = function(connection){
  var previousCounter = 0;
  var checker = setInterval(function(){
    if (previousCounter == dbQueryCounter && dbQueryCounter != 0) {
        connection.close();
        clearInterval(closeIdleDb);
    } else {
        previousCounter = dbQueryCounter;
    }
  }, maxDbIdleTime);
};

MongoClient.connect("mongodb://127.0.0.1:27017/testdb", function(err, connection)(
  if (err) throw err;
  connection.collection("mycollection").find({'a':{'$gt':1}}).toArray(function(err, docs) {
    dbQueryCounter ++;
  });   
  //do any db query, and increase the dbQueryCounter
  closeIdleDb(connection);
));

This can be a general solution for any database Connections. maxDbIdleTime can be set as the same value as db query timeout or longer.

This is not very elegant, but I can't think of a better way to do this. I use NodeJs to run a script that queries MongoDb and Mysql, and the script hangs there forever if the database connections are not closed properly.