SocketIO scaling architecture and large rooms requirements

Question

We are using socketIO on a large chat application.

At some points we want to dispatch "presence" (user availability) to all other users.

io.in('room1').emit('availability:update', {userid='xxx', isAvailable: false});

room1 may contains a lot of users (500 max). We observe a significant raise in our NodeJS load when many availability updates are triggered.

The idea was to use something similar to redis store with Socket IO. Have web browser clients to connect to different NodeJS servers.

When we want to emit to a room we dispatch the "emit to room1" payload to all other NodeJS processes using Redis PubSub ZeroMQ or even RabbitMQ for persistence. Each process will itself call his own io.in('room1').emit to target his subset of connected users.

One of the concern with this setup is that the inter-process communication may become quite busy and I was wondering if it may become a problem in the future.

Here is the architecture I have in mind.

Answer 1

Could you batch changes and only distribute them every 5 seconds or so? In other words, on each node server, simply take a 'snapshot' every X seconds of the current state of all users (e.g. 'connected', 'idle', etc.) and then send that to the other relevant servers in your cluster.

Each server then does the same, every 5 seconds or so it sends the same message - of only the changes in user state - as one batch object array to all connected clients.

Right now, I'm rather surprised you are attempting to send information about each user as a packet. Batching seems like it would solve your problem quite well, as it would also make better use of standard packet sizes that are normally transmitted via routers and switches.

Answer 2

You are looking for this library: https://github.com/automattic/socket.io-redis

Which can be used with this emitter: https://github.com/Automattic/socket.io-emitter