How can I get the IP address of a unit started on a machine and use it in another unit with fleet?

Question

I'm new to CoreOS and Docker and I'm facing with a problem with fleet.

I have a standard unit launching a POSTGRES container and I would like to know the IP address of the machine where this unit is started. I have actually a cluster of 3 machines and the POSTGRES unit isn't always started on the same machine (which means the IP is not static).

I need it when I start another unit (see below), which needs a POSTGRES.

I'm using at the moment the Unit Parameter called BindsTo :

[Unit]
Description=Test
After=docker.service
[email protected]

Requires=docker.service
[email protected]

[email protected]

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill test%i
ExecStartPre=-/usr/bin/docker rm test%i
ExecStart=/usr/bin/docker run -rm --name test%i -e HOST="HereThePostgresIP" sryther/test
ExecStop=/usr/bin/docker stop test%i

N.B. : This unit doesn't need to be started on the same machine as the POSTGRES unit.

I tried with %H variable but it returns the hostname and not the IP address of the machine.

Flannel is also used in my cluster.

Thanks!

---

I found a workaround :

[Unit]
Description=Test
After=docker.service

Requires=docker.service

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill test%i
ExecStartPre=-/usr/bin/docker rm test%i

ExecStart=/usr/bin/docker run -rm --name test%i --link postgres1:postgres sryther/test
ExecStop=/usr/bin/docker stop test%i

And the IP address is set in the container environment as POSTGRES_PORT_5432_TCP_ADDR.

Answer 1

I am using registrator for discovery in my fleet environment. When I start up my fleet I modify the cloud-config file to also include registrator (along with etcd, flannel, fleet, etc). Registrator camps on docker and recognizes when new containers are started/killed/stop. Registrator manages a database of discovered containers, it does so in several flavors (like consul, skydns). I configure mine to run in skydns mode. So, here is an example of me starting up my postgres server with a unit file, I don't know where it is going to land:

[Unit]
Description=Postgres
After=docker.service
Requires=flanneld.service docker.service etcd.service

[Service]
Restart=always
ExecStartPre=-/usr/bin/env docker kill postgresql
ExecStartPre=-/usr/bin/env docker rm postgresql
ExecStartPre=/usr/bin/env docker pull sameersbn/postgresql:9.4
ExecStart=/usr/bin/docker run --name postgresql sameersbn/postgresql:9.4
ExecStop=-/usr/bin/docker stop postgresql

When postgres starts, I get the etcd updated with a postgres entry:

/skydns/net/tacodata/postgresql/aup1:postgresql:5432

And if I fetch that one, I';; see the ip and port:

$ etcdctl get /skydns/net/tacodata/postgresql/aup1:postgresql:5432
{"host":"10.1.43.5","port":5432}

You could modify your application to do that. Or, you can run the container skydns which automatically updates dns for you, so you have an SRV record and an A record. In my installation I use a test domain called tacodata.net, so, after postgresql comes up, I have dns records!

root@f7e403be967a:/# host -t srv postgresql.tacodata.net 10.1.45.1
Using domain server:
Name: 10.1.45.1
Address: 10.1.45.1#53
Aliases: 

postgresql.tacodata.net has SRV record 10 100 5432
aup1:postgresql:5432.postgresql.tacodata.net.

When I start my dependent applications, I make them require postgresql.service, and I simply reference ip address to the application using postgresql.tacodata.net.

-g

Answer 2

According to Greg I used SkyDNS and Registrator :

I am using registrator for discovery in my fleet environment. When I start up my fleet I modify the cloud-config file to also include registrator (along with etcd, flannel, fleet, etc). Registrator camps on docker and recognizes when new containers are started/killed/stop. Registrator manages a database of discovered containers, it does so in several flavors (like consul, skydns). I configure mine to run in skydns mode.

Below you can find my working cloud-config, keep in mind that I'm using CoreOS 607 and my tests are performed on VMware. My goal was to set up 3 machines to make a small cluster with one etcd running on each machine like this.

I managed to found this solution thanks to Greg on this topic and with this cloud-config file used with Vagrant.

I hope it'll help :)

---

#cloud-config

coreos:
  etcd:
    discovery: https://discovery.etcd.io/4c8c3f9a67048e76e9075c97d9d63c0c
    addr: 192.168.1.13:4001
    peer-addr: 192.168.1.13:7001
  fleet:
    public-ip: 192.168.1.13
  units:
    - name: etcd.service
      command: start
    - name: flanneld.service
      drop-ins:
        - name: 50-network-config.conf
          content: |
            [Service]
            ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{ "Network": "172.16.0.0/16" }'
      command: start
    - name: fleet.service
      command: start
    - name: skydns.service
      enable: false
      content: |
        [Unit]
        Description=Skydns
        After=docker.service
        After=fleet.service
        [Service]
        ExecStartPre=/usr/bin/docker pull skynetservices/skydns
        ExecStart=/usr/bin/env bash -c '/usr/bin/docker run -p 53:53 --name skydns -e ETCD_MACHINES="http://$(ifconfig docker0 | awk \'/\<inet\>/ { print $2}\'):4001" skynetservices/skydns -nameservers="8.8.8.8:53" --addr="0.0.0.0:53"'
        ExecStop=/usr/bin/docker stop skydns
        Restart=always

        [X-Fleet]
        Global=true

        [Install]
        WantedBy=multi-user.target
    - name: registrator.service
      enable: false
      content: |
        [Unit]
        Description=Registrator
        After=docker.service
        After=fleet.service

        [Service]
        ExecStartPre=/usr/bin/docker pull gliderlabs/registrator
        ExecStart=/usr/bin/env bash -c '/usr/bin/docker run --net=host -p 8080:8080 -p 8443:8443 -v /var/run/docker.sock:/tmp/docker.sock --name registrator gliderlabs/registrator skydns2://$(ifconfig docker0 | awk \'/\<inet\>/ { print $2}\'):4001/skydns.local'
        ExecStop=/usr/bin/docker stop registrator
        Restart=always

        [X-Fleet]
        Global=true
    - name: primordial.service
      command: start
      content: |
        [Unit]
        Description=Load and start fleet services
        After=fleet.service
        Requires=fleet.service
        After=etcd.service
        Requires=etcd.service
        After=flanneld.service
        Requires=flanneld.service

        [Service]
        Type=oneshot
        RemainAfterExit=yes
        ExecStartPre=/usr/bin/systemctl stop docker.service
        ExecStartPre=/usr/bin/ifconfig docker0 down
        ExecStartPre=/usr/bin/brctl delbr docker0
        ExecStartPre=/usr/bin/systemctl start docker.service
        ExecStart=/usr/bin/fleetctl start /etc/systemd/system/skydns.service
        ExecStart=/usr/bin/fleetctl start /etc/systemd/system/registrator.service
ssh_authorized_keys:
  - ssh-rsa AAAA....