"connect EMFILE" error in Node.js

Question

I have very recently received a lot of traffic to my site that runs Node.js. With the increasing traffic it has started to crash a lot, which has not happened before. I get the following error in my log:

{ [Error: connect EMFILE] code: 'EMFILE', errno: 'EMFILE', syscall: 'connect' }
Error: connect EMFILE
    at errnoException (net.js:670:11)
    at connect (net.js:548:19)
    at net.js:607:9
    at Array.0 (dns.js:88:18)
    at EventEmitter._tickCallback (node.js:192:40)

Anyone that have an idea why it crash? And ideas how to solve it?

I'm using Express.js and Socket.io. It runs on Ubuntu.

Answer 1

EMFILE means error maximum files which indicates that the OS is denying your program to open more file descriptors.

Note that open files in the system include disk files, named pipes, network sockets and devices opened by all processes.

Your operating system specifies the open file limit per process.

To check open file limit of your system use ulimit -a command. Generally on unix like system it is default to 1024.

If the limitation was 1024, means you/process can open maximum 1024 files. if you exceed this limit means open, pipe and dup system calls will fail and yield EMFILE error.

When you get EMFILE it mostly indicates the leak in your code. Increasing the ulimit to a higher value is not a good solution in case there is leak in your program.

You should try to find out the cause of leak. Following can be used to debug in unix like operating system :

1. lsof meaning list open files command gives the list of open files. Assume your nodeJS program is having leak so to find out total number of file descriptors opened by your program :

   lsof | grep node | wc -l

2. To find out the file descriptors for sockets, use:

   lsof -n -i -P | grep node

Using this we can find out where the leak is and then we can correct it.

Answer 2

EMFILE error means that the OS is denying your program to open more files/sockets.

Have a look at: How do I change the number of open files limit in Linux?

Answer 3

In my case, we already had the ulimit set and kern.maxfiles configuration highest it could go on a Mac.

What fixed it was limiting the number of max sockets globally. In later versions of node the limit is Infinity.

Try adding these lines of code:

var https = require('https');
var http = require('http');

https.globalAgent.maxSockets = 5;
http.globalAgent.maxSockets = 5;

In the case of using the request library, you can configure these settings.

See: https://github.com/request/request#request---simplified-http-client

Here is more on maxSockets: https://nodejs.org/api/http.html#http_agent_maxsockets