Need for service discovery for docker engine swarm mode

Question

I'm confused about docker swarm. As far as I know, the old way to run a swarm was to run manager and workers in containers, before docker engine provided native support for swarm mode. Documentation for the old, containerized swarm explained how to setup service discovery using consul, etcd or zookeeper. Service discovery is necessary, as services are ran at random ports to avoid collisions, right?

Documentation for the docker engine swarm mode doesn't explain how to setup service discovery. Now I'm confused, if the mechanism is included in swarm mode, or is the documentation incomplete.

Where I can find clear, up-to-date explanation of swarm mode, and how it relates to concepts like service discovery?

Answer 1

While the answer given by @MagicMicky is correct, I'll try to add more context on the difference between Swarm Legacy and Swarm Mode regarding Service Discovery:

Note: I'll refer to the first version of Swarm as Swarm legacy and the new version as Swarm mode.

Service discovery with Swarm Legacy

Using Swarm Legacy, you had to deploy your own Zookeeper, Consul or Etcd to manage the cluster topology meaning nodes being assigned as Agents in the cluster. These distributed Key/Value stores were used for health monitoring and distributed locking purposes. Those were not used by Swarm to manage service discovery but only cluster node discovery and monitoring.

If you wanted Service Discovery for your containers deployed through Swarm, you had to setup an external Consul/Registrator/DNS for example and register your services on those solutions. An example on top of my mind of such a system built specifically for Swarm was Wagl.

With later versions of the docker engine (1.11), you also had access to an in-built DNS when creating overlay networks and assigning containers to an overlay network. Before 1.11, the (controversial) mechanism for service discovery was to append service entries through /etc/hosts.

In any case, overlay networking was not directly included with Swarm and this was a separate component requiring its own setup. It was more of an "add-on".

Generally the "philosophy" behind the first version of Swarm was to provide something simple and reliable to manage containers across hosts, if you needed more capabilities added to it, for example Service Discovery or Load Balancing, you had to roll your own.

---

Service Discovery with Docker Swarm Mode

As of Docker 1.12 service discovery is directly included in docker through the Swarm mode with an embedded DNS and Load Balancer. Meaning there is no need for an external component to manage Service Discovery and Load Balancing anymore.

When you create a service and assign it to an overlay, its DNS name is registered and other services part of the overlay can access it through its service name. Tasks running for a service are properly Load Balanced using the built-in LB.

For Swarm mode, the "philosophy" is more about including everything out of the box (Certificate management and rotation, service discovery, load balancing, cluster metadata through an in-built datastore, networking, scheduling) to ensure that you have the most complete system possible from day one. You are still able to swap and replace some of the components if needs be.