How to automatically update your docker containers, if base-images are updated

Question

We use a script which checks if a running container is started with the latest image. We also use upstart init scripts for starting the docker image.

    #!/usr/bin/env bash
    set -e
    BASE_IMAGE="registry"
    REGISTRY="registry.hub.docker.com"
    IMAGE="$REGISTRY/$BASE_IMAGE"
    CID=$(docker ps | grep $IMAGE | awk '{print $1}')
    docker pull $IMAGE

    for im in $CID
    do
        LATEST=`docker inspect --format "{{.Id}}" $IMAGE`
        RUNNING=`docker inspect --format "{{.Image}}" $im`
        NAME=`docker inspect --format '{{.Name}}' $im | sed "s/\///g"`
        echo "Latest:" $LATEST
        echo "Running:" $RUNNING
        if [ "$RUNNING" != "$LATEST" ];then
            echo "upgrading $NAME"
            stop docker-$NAME
            docker rm -f $NAME
            start docker-$NAME
        else
            echo "$NAME up to date"
        fi
    done

And init looks like

docker run -t -i --name $NAME $im /bin/bash

A 'docker way' would be to use docker hub automated builds. The Repository Links feature will rebuild your container when an upstream container is rebuilt, and the Webhooks feature will send you a notification.

It looks like the webhooks are limited to HTTP POST calls. You'd need to set up a service to catch them, or maybe use one of the POST to email services out there.

I haven't looked into it, but the new Docker Universal Control Plane might have a feature for detecting updated containers and re-deploying.

You can use Watchtower to watch for updates to the image a container is instantiated from and automatically pull the update and restart the container using the updated image. However, that doesn't solve the problem of rebuilding your own custom images when there's a change to the upstream image it's based on. You could view this as a two-part problem: (1) knowing when an upstream image has been updated, and (2) doing the actual image rebuild. (1) can be solved fairly easily, but (2) depends a lot on your local build environment/practices, so it's probably much harder to create a generalized solution for that.

If you're able to use Docker Hub's automated builds, the whole problem can be solved relatively cleanly using the repository links feature, which lets you trigger a rebuild automatically when a linked repository (probably an upstream one) is updated. You can also configure a webhook to notify you when an automated build occurs. If you want an email or SMS notification, you could connect the webhook to IFTTT Maker. I found the IFTTT user interface to be kind of confusing, but you would configure the Docker webhook to post to https://maker.ifttt.com/trigger/`docker_xyz_image_built`/with/key/`your_key`.

If you need to build locally, you can at least solve the problem of getting notifications when an upstream image is updated by creating a dummy repo in Docker Hub linked to your repo(s) of interest. The sole purpose of the dummy repo would be to trigger a webhook when it gets rebuilt (which implies one of its linked repos was updated). If you're able to receive this webhook, you could even use that to trigger a rebuild on your side.

One of the ways to do it is to drive this through your CI/CD systems. Once your parent image is built, have something that scans your git repos for images using that parent. If found, you'd then send a pull request to bump to new versions of the image. The pull request, if all tests pass, would be merged and you'd have a new child image based on updated parent. An example of a tool that takes this approach can be found here: https://engineering.salesforce.com/open-sourcing-dockerfile-image-update-6400121c1a75 .

If you don't control your parent image, as would be the case if you are depending on the official ubuntu image, you can write some tooling that detects changes in the parent image tag or checksum(not the same thing, tags are mutable) and invoke children image builds accordingly.

I had the same issue and thought it can be simply solved by a cron job calling unattended-upgrade daily.

My intention is to have this as an automatic and quick solution to ensure that production container is secure and updated because it can take me sometime to update my images and deploy a new docker image with the latest security updates.

It is also possible to automate the image build and deployment with Github hooks

I've created a basic docker image with that automatically checks and installs security updates daily (can run directly by docker run itech/docker-unattended-upgrade ).

I also came across another different approach to check if the container needs an update.

My complete implementation:

Dockerfile

FROM ubuntu:14.04   

RUN apt-get update \
&& apt-get install -y supervisor unattended-upgrades \
&& rm -rf /var/lib/apt/lists/*

COPY install /install
RUN chmod 755 install
RUN /install

COPY start /start
RUN chmod 755 /start

Helper scripts

install

#!/bin/bash
set -e

cat > /etc/supervisor/conf.d/cron.conf <<EOF
[program:cron]
priority=20
directory=/tmp
command=/usr/sbin/cron -f
user=root
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log
EOF

rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["/start"]

start

#!/bin/bash

set -e

echo "Adding crontab for unattended-upgrade ..."
echo "0 0 * * * root /usr/bin/unattended-upgrade" >> /etc/crontab

# can also use @daily syntax or use /etc/cron.daily

echo "Starting supervisord ..."
exec /usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf

Edit

I developed a small tool docker-run that runs as docker container and can be used to update packages inside all or selected running containers, it can also be used to run any arbitrary commands.

Can be easily tested with the following command:

docker run --rm -v /var/run/docker.sock:/tmp/docker.sock itech/docker-run exec

which by default will execute date command in all running containers and display the results. If you pass update instead of exec it will execute apt-get update followed by apt-get upgrade -y in all running containers

You would not know your container is behind without running docker pull. Then you'd need to rebuild or recompose your image.

docker pull image:tag
docker-compose -f docker-compose.yml -f production.yml up -d --build

The commands can be put in a script along with anything else necessary to complete the upgrade, although a proper container would not need anything additional.