Does a load-balancer need a load balancer?

Question

Considering a situation in which we have a web-application which is deployed in multiple servers and client requests landing to a load balancer, which in turn routes requests to actual server.

Now, if we have too many requests coming concurrently, would the load balancer itself fail? Suppose we get 1 million requests per second, won't that be beyond the processing capacity of a single load balancer?

How do we design (at least conceptually) a system which handles situations like this?

Answer 1

Putting a load balancer in front of your load balancer will not solve the problem simply because if one load balancer would failover due to the high traffic, so would the one in front!

You can achieve what you're looking for with DNS. You can register multiple IP address to a domain name and hence have multiple load balancers.

Let's say you're making a request to www.example.com. Your browser will lookup the record in the DNS and receive a list of corresponding IP addresses. Then the request will go to the first address on the list. If it's unavailable, it will go to the next on the list. The DNS servers will randomize order of the list to spread the load, and even do periodic health checks to remove unresponsive IPs. That means your requests will be split among your load balancers instead of hitting just the one.