Scaling Docker containers in the real world

Question

I have a few basic questions on scaling Docker containers:

I have 5 different apps. They are not connected to each other. Before having containers I would run 1 app per VM and scale them up and down individually in the cloud.

Now with containers I get the isolation on top of a VM, so now I can potentially run one host with 5 docker containers where each app is isolated in its own container.

As long as I have enough resources on my host I can scale up and down those containers individually as my traffic grows or shrinks. e.g. I have 3 containers running app 1, but only 1 container running app 2.

At peak times app 3 gets a lot of traffic and I need to launch a 2nd host which runs only containers for app 3.

My first question is if the above makes sense what I say or if I have misunderstood something. My second question is what technology is currently available to get this all done in an automated way. I need a load balancer and an auto scaling group which is capable of the above scenario without me having to do manual interventions.

I looked into AWS ECS and am not quite sure if it can satisfy my needs as I outlined it above.

Does anyone know how to achieve this, or is there a better way of managing and scaling my 5 apps which I am missing?

UPDATE:

Via Twitter I have been pointed to Kubernetes and specifically to the docs on the Horizontal Pod Autoscaler.

Might be useful for others as well. I will update this question as I learn more.

Answer 1

There are several options, but none that I know that does it all: you will need 2 things: autoscaling hosts according to signals, then autoscale containers on the hosts.

The following are the solutions to deploy and scale containers on the hosts (not necessarily auto-scale though):

Kubernetes is an orchestration tool which allows to schedule and (with the optional autoscaler) to autoscale pods (groups of containers) in the cluster. It makes sure your containers are running somewhere if a host fails. Google Container Engine (GKE) offers this as a service, however i am not sure they have the same functionalities to autoscale the number of VMs in the cluster as AWS does.

Mesos: somewhat similar to Kubernetes but not dedicated to running containers.

Docker Swarm: the Docker multi-host deployment solution, allows you control many hosts as if they were a single Docker host. I don't believe it has any kind of 'autoscaling' capability, and I don't believe it takes care of making sure pods are always running somewhere: it's basically docker for cluster.

[EDIT] Docker supports restarting failing containers with the restart=always option, also, as of Docker 1.11 Docker Swarm is a mode in Docker Daemon, and supports rescheduling containers on node failure: it will restart containers on a different node if a node is no longer available.

Docker 1.11+ is becoming a lot like Kubernetes in terms of functionalities. It has some nice features (like TLS between nodes by default), but still lacks things like static IPs and storage provisioning

None of these solutions will autoscale the number of hosts for you, but they can scale the number of containers on the hosts.

For autoscaling hosts, solutions are specific to your cloud provider, so these are dedicated solution. The key part for you is to integrate the two: AWS allows deployment of Kubernetes on CoreOS; I don't think they offer this as a service, so you need to deploy your own CoreOS cluster and Kubernetes.

Now my personal opinion (and disclaimer)

I have mostly used Kubernetes on GKE and bare-metal, as well as Swarm a about 6 months ago, and i run an infra with ~35 services on GKE:

Frankly, GKE with Kubernetes as a Service offers most of what you want, but it's not AWS. Scaling hosts is still a bit tricky and will require some work.

Setting up your own Kubernetes or Mesos on AWS or bare metal is very feasible, but there is quite a learning curve: it all depends if you really strongly feel about being on AWS and are willing to spend the time.

Swarm is probably the easiest to get working with, but more limited, however homebuilt script can well do the job core job: use AWS APIs to scale hosts, and Swarm to deploy. The availability guarantee though would require you monitoring and take care of re-launching containers if a node fails.

Other than that, there are also container hosting providers that may do the job for you:

• Scalingo is one i know of but there are others. https://scalingo.com/

• OVH Sail Above has this service in alpha. https://www.runabove.com/sailabove.xml