JavaScript leaking memory (Node.js/Restify/MongoDB)

Question

Update 4: By instantiating the restify client (see controllers/messages.js) outside of the function and calling global.gc() after every request it seems the memory growth rate has been reduced a lot (~500KB per 10secs). Yet, the memory usage is still constantly growing.

Update3: Came across this post: https://journal.paul.querna.org/articles/2011/04/05/openssl-memory-use/

It might be worth noting that I'm using HTTPS with Restify.

Update 2: Updated the code below to the current state. I've tried swapping out Restify with Express. Sadly this didn't make any difference. It seems that the api call at the end of the chain (restify -> mongodb -> external api) causes everything to retain to memory.

Update 1: I have replaced Mongoose with the standard MongoDB driver. Memory usage seems to grow less fast, yet the leak remains..

---

I've been working on trying to locate this leak for a couple of days now.

I'm running an API using Restify and Mongoose and for every API call I do at least one MongoDB lookup. I've got about 1-2k users that hit the API multiple times in a day.

What I have tried

• I've isolated my code to just using Restify and used ApacheBench to fire a huge amount of requests (100k+). The memory usage stays around 60MB during the test.
• I've isolated my code to just using Restify and Mongoose and tested it the same way as above. Memory usage stays around 80MB.
• I've tested the full production code locally using ApacheBench. Memory usage stays around 80MB.
• I've automatically dumped the heap on intervals. The biggest heap dump I had was 400MB. All I can see that there are tons of Strings and Arrays but I cannot clearly see a pattern in it.

So, what could be wrong?

I've done the above tests using just one API user. This means that Mongoose only grabs the same document over and over. The difference with production is that a lot of different users hit the API meaning mongoose gets a lot of different documents.

When I start the nodejs server the memory quickly grows to 100MB-200MB. It eventually stabilizes around 500MB. Could this mean that it leaks memory for every user? Once every user has visited the API it will stabilize?

I've included my code below which outlines the general structure of my API. I would love to know if there's a critical mistake in my code or any other approach to finding out what is causing the high memory usage.

Code

app.js

var restify = require('restify');
var MongoClient = require('mongodb').MongoClient;

// ... setup restify server and mongodb

require('./api/message')(server, db);

api/message.js

module.exports = function(server, db) {

    // Controllers used for retrieving accounts via MongoDB and communication with an external api
    var accountController = require('../controllers/accounts')(db);        
    var messageController = require('../controllers/messages')();

    // Restify bind to put
    server.put('/api/message', function(req, res, next) {
        // Token from body
        var token = req.body.token;

        // Get account by token
        accountController.getAccount(token, function(error, account) {

            // Send a message using external API
            messageController.sendMessage(token, account.email, function() {
                res.send(201, {});
                return next();
            });
        });
    });
};

controllers/accounts.js

module.exports = function(db) {

    // Gets account by a token
    function getAccount(token, callback) {
        var ObjectID = require('mongodb').ObjectID;

        var collection = db.collection('accounts');

        collection.findOne({
            token: token
        }, function(error, account) {

            if (error) {
                return callback(error);
            }

            if (account) {
                return callback('', account);
            }

            return callback('Account not found');
        });
    }
};

controllers/messages.js

module.exports = function() {

    function sendMessage(token, email, callback) {

        // Get a token used for external API
        getAccessToken(function() {}

            // ... Setup client

            // Do POST
            client.post('/external_api', values, function(err, req, res, obj) {
                return callback();
            });

        });
    }

    return {
        sendMessage: sendMessage
    };
};

Heap snapshot of suspected leak

Answer 1

Might be a bug in getters, I got it when using virtuals or getters for mongoose schema https://github.com/LearnBoost/mongoose/issues/1565