Scalable architecture for socket.io

Question

I am new to socket.io and Node JS and I am trying to build a scalable application with a high number of simultaneous socket connections (10,000+).

Currently, I started on a model where my server creates child process, and every child process listens a specific port with a sicket.io instance attached. Once a client connects, he is redirected on a specific port.

The big question is : Does having several socket.io instances on several ports increases the number of possible connections ?

Here is my code, just in case :

Server

var server = http.createServer(app);

server.childList = [];
for (var i = 0; i < app.portList.length; i++) {
  server.childList[i] = require('child_process').fork('child.js');
}

server.listen(3443, () => {
  for (var i = 0; i < app.portList.length; i++) {
    server.childList[i].send({ message: 'createServer', port: app.portList[i] });;
  }
});

child.js :

var app = require('./app');
var http = require('http');
var socket_io        = require( "socket.io" );

process.on('message', (m) => {
    if (m.message === 'createServer') {

        var childServ = http.createServer(app);

        childServ.listen(m.port, () => {
            console.log("childServ listening on port "+m.port);
        });

        var io = socket_io();
        io.attach( childServ );

        io.sockets.on('connection', function (socket) {
            console.log("A client just connected to my socket_io server on port "+m.port);
        });
    }
});

Feel free to release the kraken if I did something horrible there

Answer 1

First off, what you need to optimize depends on how busy your socket.io connections are and whether the activity is mostly asynchronous I/O operations or whether it's CPU-intensive stuff. As you may already know, node.js scales really well already for asynchronous I/O stuff, but it needs multiple processes to scale well for CPU-intensive stuff. Further, there are some situations where the garbage collector gets too busy (lots and lots of small requests being served) and you also need to go to multiple processes for that reason.

More server instances (up to at least the number of CPUs you have in the server) will give you more CPU processing power (if that's what you need). It won't necessarily increase the number of max connections you can support on a box if most of them are idle. For that, you have to custom tune your server to support lots and lots of connections.

Usually, you would NOT want N socket.io servers each listening on a different port. That puts the burden on the clients to somehow select a port and the client has to know exactly what ports to choose from (e.g. how many server instances you have).

Usually, you don't do it this way. Usually, you have N processes all listening on the same port and you use some sort of loadbalancer to distribute the load among them. This makes the server infrastructure transparent to the clients which means you can scale the servers up or down without changing the client behavior at all. In fact, you can even add more than one physical server box and increase capacity even further that way.

Here's an article from the socket.io doc on using multiple nodes with a load balancer to increase capacity: Socket.io - using multiple nodes (updated link). There's also explicit support by redis for a combination of multiple socket.io instances and redis so you can communicate with any socket.io instance regardless of process.

Answer 2

Does having several socket.io instances on several ports increases the number of possible connections ?

Yes, you have built a simple load-balancer which is a pretty common practice. There are several good tutorials about different ways of scaling node.js.

• Horizontally scale socket.io with redis
• http://goldfirestudios.com/blog/136/Horizontally-Scaling-Node.js-and-WebSockets-with-Redis

Your load balancer will speed up your code to a point because you utilize multiple threads but I read on some other thread a while ago that a rule of thumb is to start around 2-3 processes per cpu core. More than that cause more overhead then help, but that is highly dependent on situation.