Load balancer - how to write one for a custom application?

Question

I've written a simple server application which will run distributed on several machines.

My question is how does a network load balancer works, in general?

I've heard of round-robin and other algorithms, but what I haven't got answer to is how does the process really goes? In socket terms.

The client connects to one of the load balancer machines, asks for a "free-to-connect-to" server and simply connects to it?
That's the simpliest way I can think of.
.. or, does it use the load balancer as a proxy (that implies that all the NBs must be always connected to the application servers, and data is transferred through them)?

It's more of a general question. How would you do this?

Thank you all!

Answer 1

There are several different ways to load balance an application. Some are physical devices that sit between your router and the servers; some are software based with a bit of code that runs on each of the load balanced devices.

Microsoft has load balancing built into Windows which is all software based. It's pretty good and easy to set up.

However, I'll cover the physical route.

There are several algorithms here, but the main one is Round Robin with an option for "sticky" sessions. Sticky in this case means that the load balancer will try to keep a history of clients and forward requests from the same client to the same machine. This means the load balancer needs to keep a list of clients and where it directed those clients. Depending on cache size, clients may fall off the list and on future requests they may be forwarded to a different server.

Round Robin is a pretty simple idea. For each request that comes in send it to the next server in the list. More complicated algorithms might take into account how many requests go to a particular server and how long are those requests taking; then try to rebalance new requests to favor faster servers. This part is complicated though.