Node.js not closing files created by fs.createReadStream()

Question

On my server, every time a user uses our service we have to grab a JSON file for them from the server. I do this by using fs.createReadStream() inside of my own function.

function getJSONFromServer(filepath, callback){
    var data = fs.createReadStream(filepath);
    data.on('error', function (error) {
        console.log("Caught", error);
        callback(undefined, error);
    });
    var jsonFile = "";
    data.on('data', function(chunk) {
        jsonFile += chunk;
    });
    data.on('end', function() {
        var jsonData = JSON.parse(jsonFile);
        callback(jsonData);
        data.destroy();
        data.close();
    });
}

This does the job, but it does not close the connection to the file. So after reading 1024 files (the limit on my server), Node.js will then produce the error EMFILE, too many open files. Then I have to kill our Node.js server, open it again and that will clear the "open files".

I check the amount of files open by lsof -i -n -P | grep nodejs. It displays something like this:

nodejs  13707     node   10u  IPv4 1163695      0t0  TCP 127.0.0.1:55643->127.0.0.1:27017 (ESTABLISHED)
nodejs  13707     node   11u  IPv4 1163697      0t0  TCP 127.0.0.1:55644->127.0.0.1:27017 (ESTABLISHED)

for as many files that are open.

I've tried using graceful-fs. I've tried calling stream.destroy() and stream.close(), but I still get the same issue. My server is essentially a ticking time bomb because we get a heavy, steady flow of users and after so many users have connected it will just stop working.

Also, ulimit -n [open file amount] does not work, and even if it did, this is not a long term solution because I'd like my file connections to close and not sit open for no reason.

I'm using Node.js version v0.10.25, Ubuntu 15.04 (GNU/Linux 3.19.0-42-generic x86_64) and the latest version of graceful-fs if that helps!

Thanks for any help you can provide.

Answer 1

This has got to be the stupidest mistake I've ever made. Regardless, here's the answer. I hope I can save someone from dealing with this error and almost ripping their hair out.

I was running my app with nodejs and not node. Turns out, if you do nodejs --version, it will likely return a version that is very old, which was v0.10.25 for me. node --version however was v5.6.0. Obviously this massive jump in versions would fix some stuff, so I ran the app with node app.js instead of nodejs app.js and I haven't had the issue at all since. There are now only 6 open files, whereas before we had over 1000 with time.

Damn it feels good to have this off my chest.