How to run Consul on docker with initial key-value pair data?

Question

I am trying to spin a Consul server on docker container and use it as config server for my SpringBoot cloud application. For that I want to have some pre-configured data(Key-Value pairs) in Consul.

My current config in docker-compose.yml is:

  consul:
    image: "progrium/consul:latest"
    container_name: "consul"
    ports:
      - '9330:8300'
      - '9400:8400'
      - '9500:8500'
      - '9600:53'
    command: "-server -bootstrap -ui-dir /ui"

Is there a way to pre-populate key-value pairs?

Answer 1

Compose

Compose doesn't have "tasks" as built in concept, but you can set them up with multiple compose files in a project. A docker-compose-init.yml could define the tasks, rather than long running services but you need to manage orchestration yourself. I've put an example on my consul demo.

docker-compose up -d
docker-compose -f docker-compose-init.yml run consul_init

Image Build

You can add image build RUN steps to add the data. The complication here is running the server the same way you normally would, but in the background, and adding the data all in the one RUN step.

FROM progrium/consul:latest
RUN set -uex; \
    consul agent -server --bootstrap -data-dir /consul/data & \ 
    let "timeout = $(date +%s) + 15"; \
    while ! curl -f -s http://localhost:8500/v1/status/leader | grep "[0-9]:[0-9]"; do\
      if [ $(date +%s) -gt $timeout ]; then echo "timeout"; exit 1; fi; \
      sleep 1; \
    done; \
    consul kv put somekey somevalue;

Image Startup

Some databases add a script to the image to populate data at startup. This is normally so users can control setup via environment variables injected at run time, like mysql/postgres/mongo.

FROM progrium/consul:latest
ENTRYPOINT my-entrypoint.sh

Then your script starts the server, sets up the data, and then at the end continues on as the image would have before.

Answer 2

Here a very similar approach but maybe simpler and it works. Does not require compose, just Docker and all is done in the image.

This directory structure:

bootstrap/values.json
bootstrap/start.sh
bootstrap/init.sh
Dockerfile

Dockerfile

FROM consul

RUN mkdir /tmp/bootstrap

COPY bootstrap/* /tmp/bootstrap/
RUN chmod 755 /tmp/bootstrap/*
RUN dos2unix /tmp/bootstrap/*

CMD /tmp/bootstrap/start.sh

bootstrap/start.sh

#!/bin/sh

/tmp/bootstrap/init.sh &

consul agent -dev -client=0.0.0.0

bootstrap/init.sh

#!/bin/sh

echo "bootstrap values - wait until port is available"

while ! nc -z localhost 8500; do   
  sleep 1
done

echo "executing consul kv command"

consul kv import @/tmp/bootstrap/values.json

bootstrap/values.json

[
    {
            "key": "config/",
            "flags": 0,
            "value": ""
    },
    {
            "key": "config/yoursoftware/",
            "flags": 0,
            "value": ""
    },
]

The main challenge is that RUN can not process background tasks and CMD is the only main executable which must run at the end. So background processes can be started in via CMD, which uses here a shell script (start.sh). In that script the ampersand symbol gives under linux a way to start a process and throw it to background. It will execute the init.sh. In that it must now wait until the port is ready and then it can use the import.