How to load balance services in CoreOS?

Question

• I have a coreos cluster running 3 web conntainers, each container running the same web application.
• How do I load balance my requests to these running containers using haproxy and nginx.
• Please suggest.

Answer 1

I would suggest you take a close look at vulcand, which is a reverse proxy running on coreos that is designed to load balance http requests across containers running on coreos:

Vulcand is a reverse proxy for HTTP API management and microservices. It is inspired by Hystrix.

It uses Etcd as a configuration backend, so changes to configuration take effect immediately without restarting the service.

https://coreos.com/blog/zero-downtime-frontend-deploys-vulcand/

http://vulcand.github.io/

A good blog post that includes using vulcand for load balancing across docker containers:

http://www.recorditblog.com/post/how-to-create-a-web-scale-infrastructure-based-on-docker-coreos-vulcand-and-mesos-and-why-object-storage-becomes-the-de-facto-data-repository/

This github repo has some sample source code for setting up coreos, docker and vulcand:

https://github.com/bradgignac/intro-to-coreos

Answer 2

This is a little late but it is really similar to what the digital ocean tutorial teaches you to do in their Getting Started with CoreOS:

https://www.digitalocean.com/community/tutorials/how-to-use-confd-and-etcd-to-dynamically-reconfigure-services-in-coreos

Pretty much you run the HAProxy container on one machine and have an arbitrary number of services running backend servers. Then you use confd to build the config file dynamically as you add and remove backend servers.

This is my HAProxy Service that I built based on the tutorials. I call it alongside my Webapp using these two service files.

Note this line in my webapp.service file:

ExecStartPost=/bin/bash -c 'etcdctl set /app/servers/%n "%n $(curl -sw "\n" http://169.254.169.254/metadata/v1/interfaces/private/0/ipv4/address):80"'

This adds the server location to be used in the HAProxy config file to /app/servers/webapp.service. The Digital Ocean tutorial does this via a 'Sidekick' container which may be more versatile and less error prone however.