"Topology was destroyed" when using MongoDB with native driver and Express.js

Question

I have implemented simple application that fetches data from MongoDB

const express = require('express')
const app = express()
const port = 3000

const MongoClient = require('mongodb').MongoClient;
const assert = require('assert');

const dbConnectionURL = 'mongodb://localhost:27017';
const dbName = 'todo';
const dbClient = new MongoClient(dbConnectionURL);

function findTodos(db, callback) {
    const collection = db.collection('todos');
    collection.find({}).toArray(function (err, todos) {
        assert.equal(err, null);
        console.log("Found the following records");
        console.log(todos)
        callback(todos);
    });
}

app.get('/', (req, res) => {
    dbClient.connect(function (err) {
        assert.equal(null, err);
        console.log("Connected successfully to server");

        const db = dbClient.db(dbName);

        findTodos(db, function(todosArr) {
            var todos = { todos: todosArr }
            res.send(todos)
            dbClient.close()
        });
    });
});

app.listen(port, () => console.log(`Example app listening on port ${port}!`))

It is basically copy-pasted from online tutorials. First time I make http request to 'http://localhost:3000/' it works. But second time I get

todo-backend/node_modules/mongodb/lib/utils.js:132
    throw err;
    ^

AssertionError [ERR_ASSERTION]: 'MongoError: Topology was destroyed' == null
    at todo-backend/index.js:15:16
    at err (todo-backend/node_modules/mongodb/lib/utils.js:415:14)
    at executeCallback (todo-backend/node_modules/mongodb/lib/utils.js:404:25)
    at handleCallback (todo-backend/node_modules/mongodb/lib/utils.js:128:55)
    at cursor._endSession.cursor._endSession (todo-backend/node_modules/mongodb/lib/operations/cursor_ops.js:207:38)
    at ClientSession.endSession (todo-backend/node_modules/mongodb-core/lib/sessions.js:129:41)
    at Cursor._endSession (todo-backend/node_modules/mongodb-core/lib/cursor.js:189:13)
    at Cursor._endSession (todo-backend/node_modules/mongodb/lib/cursor.js:226:59)
    at cursor._next (todo-backend/node_modules/mongodb/lib/operations/cursor_ops.js:207:20)
    at initializeCursor (todo-backend/node_modules/mongodb-core/lib/cursor.js:766:16)

I have looked over the Internet for answer and found similar error but when people are doing more complicated things. What I did seems for more very basic and I can not find any room for change that might help.

If I remove dbClient.close() it works but then I see in MongoDB logs that number of connections is constantly growing. Of course I can introduce a flag and check if I already connected. But with this question I want to understand a root cause.

Answer 1

This is because the code contains an anti-pattern: every time a new request comes in, it opens a new database connection, then closes that connection once the response was sent. It then subsequently tried to reuse the closed connection, hence the error message you're seeing on the 2nd request.

What you want is to connect only once to the database for the lifetime of the application using a global connection object, then use that global object to perform your database operations.

Using this global object allows the MongoDB driver to properly create a connection pool to the database. This pool is managed by the MongoDB driver, and avoids the expensive connect/reconnect pattern.

For example:

// listen on this port
const port = 3000

// global database client object
var client = null

// listen on the configured port once database connection is established
MongoClient.connect('mongodb://localhost:27017', { useNewUrlParser: true }, (err, res) => {
  assert.equal(null, err)
  client = res
  app.listen(port, () => console.log(`Example app listening on port ${port}!`))
})

// use the client global object for database operations
app.get('/', (req, res) => {
  db = req.query.db
  col = req.query.col
  client.db(db).collection(col).find({}).toArray((err, docs) => {
    assert.equal(null, err)
    res.send(JSON.stringify(docs))
  })
})

Edit to answer your question in the comment:

Why does it tries to reuse previous connection when I do connect every time?

This is because in the original code, dbClient was globally defined. When dbClient.close() was called, the global dbClient was closed. An error was then produced when that dbClient object was reused. This is because connect() creates a connection pool instead of a single connection, and was not expected to be called multiple times per invocation.

If you move the dbClient variable from the global scope into the app.get() context, you'll find that no error will be produced when you call the HTTP endpoint multiple times, as a new dbClient object was created every time.

Having said that, although it will work, this is not a recommended pattern. It's better to use a pattern similar to the example code I posted above.