Docker-compose: deploying service in multiple hosts

Question

I have a docker-compose file that deploys 8 different docker services in the same host. Is it possible to deploy it in different hosts?, I would like to deploy some service in one hosts and another ones in other host remotelly. Would I need to use docker-swarm? or is an easier way to do?

I have read that it could be done using DOCKER_HOST, but if I configure /etc/default/docker with this variable, all the services would be run on the remote host, and what I need is some services in one remote host, and other services in other remote host.

Answer 1

• We can do this with docker compose v3 now.

  https://docs.docker.com/engine/swarm/#feature-highlights https://docs.docker.com/compose/compose-file/

• You have to initialize the swarm cluster using command

  $ docker swarm init

• You can add more nodes as worker or manager -

  https://docs.docker.com/engine/swarm/join-nodes/

• Once you have your both nodes added to the cluster, pass your compose v3 i.e deployment file to create a stack. Compose file should just contain predefined images, you can't give a Dockerfile for deployment in Swarm mode.

  $ docker stack deploy -c dev-compose-deploy.yml --with-registry-auth PL

• View your stack services status -

  $ docker stack services PL

• Try to use Labels & Placement constraints to put services on different nodes.

Example "dev-compose-deploy.yml" file for your reference -

version: "3"

services:

  nginx:
    image: nexus.example.com/pl/nginx-dev:latest
    extra_hosts:
      - "dev-pldocker-01:10.2.0.42”
      - "int-pldocker-01:10.2.100.62”
      - "prd-plwebassets-01:10.2.0.62”
    ports:
      - "80:8003"
      - "443:443"
    volumes:
      - logs:/app/out/
    networks:
      - pl
    deploy:
      replicas: 3
      labels:
        feature.description: “Frontend”
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: any
      placement:
        constraints: [node.role == worker]
    command: "/usr/sbin/nginx"

  viz:
    image: dockersamples/visualizer
    ports:
      - "8085:8080"
    networks:
      - pl
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    deploy:
      replicas: 1
      labels:
        feature.description: "Visualizer"
      restart_policy:
        condition: any
      placement:
        constraints: [node.role == manager]

networks:
pl:

volumes:
logs:

Answer 2

With docker swarm mode, you can deploy a version 3 compose yml file using:

docker stack deploy -c docker-compose.yml $your_stack_name

The v3 syntax removes a few features that do not apply to swarm mode, like links and dependencies. You should also note that volumes are stored local to a node by default. Otherwise the v3 syntax is very similar to the v2 syntax you may already be using. See ether following for more details:

https://docs.docker.com/compose/compose-file/

https://docs.docker.com/engine/swarm/

---

[ Original answer before v3 of the docker-compose.yml ]

For a single docker-compose.yml to deploy to multiple hosts, you need to use the standalone swarm (not the newer swarm mode, yet, this is rapidly changing). Spin up a swarm manager that has each host defined as members of its swarm, and then you can use constraints inside your docker-compose.yml to define which services run on which hosts.

You can also split up your docker-compose.yml into several files, one for each host, and then run multiple docker-compose up commands, with a different DOCKER_HOST value defined for each.

In both cases, you'll need to configure your docker installs to listen on the network, which should be done by configuring TLS on those sockets. This documentation describes what you need to do for that.

Answer 3

You can use docker compose version 3 which provides ability to do multi host deployment without using multiple compose files. All you need is to define labels for each node in the cluster and use the label name for placement constraint.

Answer 4

You may also want to consider the Overnode tool - it is container orchestration tool on top of automated multi-host docker-compose. It is the easiest transition from the single-host docker-compose deployment. (Disclaimer: I am an author and it was published recently)