Scale Socket.io vertically AND horizontally - what is the "right" way to go?

Question

I want to scale my Node.js Socket application vertically and horizontally and I haven´t found a sophisticated solution yet.

My application has two use-cases:

1. Broadcast messages from one user to all others
2. Push messages from one user to a subset of users

---

On one hand, I´ve read that I need Redis for both cases together with socket.io-redis

On the other hand, I´ve watched this video and read this SO answer where it says that Redis isn´t reliable and it´s not guaranteed that the published messages will arrive, so you should only use it for clustering/vertical scaling

Microsoft Azures solution to use ServiceBus is out of question, because I don´t want to use Azure.

Instead of Redis, the guy recommends using RabbitMQ for horizontal scaling.

For the vertical scaling there is also socket.io-clusterhub, an IPC for node processes, but it seems to work only on Socket.io <= v0.9.0

Then there is this guy, who has implemented his own method to pass messages to other nodes via HTTP requests, which makes somehow sense. But why HTTP requests if you could also establish direct socket connections between servers, push the message to all servers simultaneously and overcome the delay of going from one server to another?

---

As a conclusion I thought maybe I could go with Redis on EACH server, just for the exchange of messages when clustering my application on multiple processes, together with RabbitMQ as a S2S communication solution.

But it seems a bit like an overkill to have one Redis per Server and another central RabbitMQ.

Is there any known shorter/better solution to scale Socket.io reliably in both directions?

---

EDIT: I´ve tried using a single Redis Server for multiple Node.js Servers, where each of them uses Clustering via sticky-session over all cores. While the Clustering at its own works like a charm with redis, there seems to be a problem when using multiple servers. Messages won´t arrive at the other nodes.

Answer 1

I'd say Kafka is a good fit for the horizontal scaling. It is a fairly sophisticated way of distributing a huge amount of events across servers (which at the end is what you want). This is a good read about it: https://engineering.linkedin.com/kafka/running-kafka-scale

Regarding the vertical scale, instead of socket.io-clusterhub I would use something called PM2 (https://github.com/Unitech/pm2) which allows you to resize the scale of the apps in every computer dynamically as well as controlling the logs and reporting to keymetrics.io (if you are using it).

If you need any snippets ask me and I will edit the answer but in the PM2 github there are quite few.