Node.js Multi Server Clustering

Question

I'm working on a project with Node.js that involves a server (for the sake of simplicity let's imagine this server as a chat server which have to forward messages from certain clients to other clients). I need for QoS reasons that this server is always reachable, so I thought to use clustering to divide the balance load between different servers (different physical machines) and to be sure that if a server goes down, another will be ready to serve the requests.

My question is: is this kind of distributed approach possible in Node.js?

I already read about the "cluster" module, but, from what I understood, it seems to scale only on multiprocessors on the same machine.

Answer 1

Yes it is possible.

It is not a property of NodeJS but rather the architecture you design for your application which will determine whether you can do it or not.

Your main problem will always be to share state across your instances, so say you have 4 chat servers A B C D, and you have a LoadBalancer L which spreads connections across the 4 servers, then when A goes down, and you reconnect all A's connections to the remaining instances, how do you ensure that the state of the chatroom is the same on B C and D?

One way is to have the application code be completely stateless, and push all data to a distributed in memory database, like mongoDb or Redis. You want the database to be distributed in case one of the database instances go down.

Now you last problem is the LoadBalancer. If that goes down, your entire system is down.

So to make a long story short; Yes you can do it, but you need to make some hard decisions on where your potential points of failure are going to be. If you want no single point of failure, you are going to need a complex and expensive setup.

Answer 2

My recommendations is that you leverage independent clusters to share state. In other words, have an API cluster where servers dont' talk with each other, but instead share a common Redis instance/cluster, and share a common MongoDB cluster. This allows you to share sessions, variables, and you can leverage pub/sub capabilities of Redis to avoid needing any gossiping within your API cluster.

For Chat specifically, if you use Redis with Socket.IO as your client, then whenever you broadcast to a lobby, it'll use redis behind the scenes to broadcast that message to that lobby, despite lobby members existing on multiple servers. Additionally, this creates another level of fault tolerance as any server can manage socket reconnections, and socket.io will automatically reconnect to the API cluster if it's connection is broken, all while maintaining state via Redis.