Aws load balancer for Server Sent Events or Websockets

Question

I'm trying to load balance a nodejs Server sent event backend and I need to know if there is a way to distribute the new connections to the instances with the least connected clients. The problem I have is when scaling up, the routing continues sending new connections to the already saturated instance and since the connections are long lived this simply won't work.

What options do I have for horizontal scaling long lived connections?

Answer 1

It looks like you want a Load Balancer that can provide both "sticky sessions" and use the "least connection" instead of "round-robin" policy. Unfortunately, NGINX cannot provide this.

HAProxy (High Availability Proxy) allows for this:

backend bk_myapp
 cookie MyAPP insert indirect nocache
 balance leastconn
 server srv1 10.0.0.1:80 check cookie srv1
 server srv2 10.0.0.2:80 check cookie srv2

If you need ELB functionality and want to roll it all manually, take a look at this guide.

You might also want to make sure classic AWS ELB "sticky session" configuration or the newer ALB "sticky session" option does not meet your needs. ELB normally sends connection to upstream server with the least "load", and when combining with sticky sessions might be enough.