How to distribute persistent connection across a Elixir/Phoenix cluster?

Question

So this question is bugging me, and I actually don't need this kind of scale, I'm just curious. Recently Chris Mccord got 2 million persistent connections in a single box using Elixir/Phoenix. If I have 3 boxes in a cluster, each one handling 2 million persistent connections:

1. Does that mean that the load balancer has to keep 6 million persistent connections or is it possible to just forward the connection to the backend server and close the connection on the load balancer?
2. Do I have to distribute the connections using some logic in the client and avoid a single entry point? Something like box1.foo.com, box2.foo.com, box3.foo.com and tell the client to connect to one of them?

I know the answer can be application specific, but can you please give some examples on how to achieve this kind of scale? Again, this is just a thought exercise, I couldn't find the answer anywhere and I'm interested in horizontal scaling.

Thank you.

Answer 1

you'll want to use multiple load balancers (or skip the load balancer entirely) and use round robin dns (or weighted dns) to distribute requests across them. this is how heroku's routing mesh and aws' elastic load balancers work. you could also spend $$$ on something like an f5 load balancer