Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

docker-compose scaling with unique environment variable

I have a sample compute service in my docker-compose file which works just great as expected.

version: "3"
services:
  compute-service:
    image: dummy/compute
    environment:
      - INPUT=2

However there could be times in which I need to run this service with diff inputs (say INPUT = 4, 7, 9, 10, 12..etc). I do not like the idea of copying and pasting the service multiple times for each input. Scaling is an option. But how can I ensure that each instance works on unique input variable.


I am aware that I could use an env variable like this. My question is rather related to how to pass unique values as part of scaling!!

version: "3"
services:
  compute-service:
    image: dummy/compute
    environment:
      - INPUT=${INPUT}
like image 827
KitKarson Avatar asked May 18 '19 22:05

KitKarson


People also ask

Can you use environment variables in Docker compose?

Docker Compose allows us to pass environment variables in via command line or to define them in our shell. However, it's best to keep these values inside the actual Compose file and out of the command line. “Why?” you may ask.

How do I scale a docker compose file?

This can be controlled by assigning port range on the ports section of the compose YAML file. Scaling can also be done by using up command as well with the --scale flag. Alternatively, in Compose file version 3. x, you can also specify replicas under the deploy section as part of a service configuration for Swarm mode.

How do I override Docker compose environment variables?

Overriding a single value in your docker-compose . env file is reasonably simple: just set an environment variable with the same name in your shell before running your docker-compose command.

Can Docker compose auto scale?

Docker – more specifically Docker-Compose – provides a solution with auto-scaling groups.


Video Answer


1 Answers

With docker-compose, I don't believe there's any support for this. However, with swarm mode, which can use a similar compose file, you can pass {{.Task.Slot}} as an environment variable using service templates. You can deploy a single node swarm cluster with docker swarm init. Instead of docker-compose up, I'm deploying the following example with docker stack deploy -c docker-compose.yml test.

And here's an example docker-compose.yml file using the {{.Task.Slot}} functionality:

version: '3'
services:
  test:
    image: busybox
    command: /bin/sh -c "echo My task number is $$task_id && tail -f /dev/null"
    environment:
      task_id: "{{.Task.Slot}}"
    deploy:
      replicas: 5

Then, reviewing each of these running containers:

$ docker ps --filter label=com.docker.swarm.service.name=test_test
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES
ccd0dbebbcbe        busybox:latest      "/bin/sh -c 'echo My…"   About a minute ago   Up About a minute                       test_test.3.i3jg6qrg09wjmntq1q17690q4
bfaa22fa3342        busybox:latest      "/bin/sh -c 'echo My…"   About a minute ago   Up About a minute                       test_test.5.iur5kg6o3hn5wpmudmbx3gvy1
a372c0ce39a2        busybox:latest      "/bin/sh -c 'echo My…"   About a minute ago   Up About a minute                       test_test.4.rzmhyjnjk00qfs0ljpfyyjz73
0b47d19224f6        busybox:latest      "/bin/sh -c 'echo My…"   About a minute ago   Up About a minute                       test_test.1.tm97lz6dqmhl80dam6bsuvc8j
c968cb5dbb5f        busybox:latest      "/bin/sh -c 'echo My…"   About a minute ago   Up About a minute                       test_test.2.757e8evknx745120ih5lmhk34

$ docker ps --filter label=com.docker.swarm.service.name=test_test -q | xargs -n 1 docker logs
My task number is 3
My task number is 5
My task number is 4
My task number is 1
My task number is 2
like image 55
BMitch Avatar answered Sep 25 '22 02:09

BMitch