How to run WebSockets with multiple server instances?

Question

It is useful to run multiple instances of HTTP servers for scaling.

However, it seems that this would not work with WebSockets because each server instance would have its own set of connections to clients.

How can you run WebSockets on multiple server instances if they all need to share a common set of connections?

Answer 1

I was looking for the same thing to develop websocket app ready to scale, and apparently not much article discuss about this.

Case 1 The recommended approach is using Redis cache like Heroku suggest, use Service Bus or store into db like the 3 approaches mentioned in Scaleout in SignalR. However there is a catch, there must be a slight latency because in order the second instance want to know, scheduler or background worker/service should be run continuously to detect new changes in the cache/queue/db.

Case 2 In my case I want high frequency nearly real time. So it should have a persistence connection(s) between instances. However the challenge, we can't do that between stateless web server.

So we still need intermediate long running background/worker service and implement our own net/socket/TCP from code and maintain the connection pool but that is not an easy task.

I am thinking of using Websocket client on server (for this service) so the pattern to communicate with client and between instances is almost the same. See this: 1 2. One thing left is to update the instance IP/URL into distributed cache or db on every instance up/down.

Last option is to explore on a new WebRTC

Answer 2

I havn't tried it yet but i am thinking of making connections sticky may solve the problem. You can set this setting in your load balancer.