Docker image fails to build on Live but fine on Dev

Question

I have a strange issue with Docker.

This is the Dockerfile in question.

FROM python:2.7

RUN apt-get update && apt-get install -y \
    build-essential \
    python-lxml \
    python-dev \
    python-pip \
    python-cffi \
    libcairo2 \
    libpango1.0-0 \
    libpangocairo-1.0.0 \
    libxml2-dev \
    libxslt1-dev \
    zlib1g-dev \
    libpq-dev \
    libjpeg-dev \
    libgdk-pixbuf2.0-0 \
    libffi-dev \
    mysql-client \
    shared-mime-info

# ... further docker file rules, which doesn't get run cause apt-get fails

The problem I'm having is that on my development machine, this Dockerfile builds perfectly fine, but on our live servers it's suddenly failing (worked in the past), with E: Package 'mysql-client' has no installation candidate.

I thought the point of Docker is that everything runs using the same image and that you shouldn't run into issues like this.

Why is this the case and what can I do to fix it from here so that it runs the same on both dev and live?

Answer 1

You are using image python with tag 2.7, however this tag is a "shared" tag as per Python readme on Docker Hub which is changing other time: right now python:2.7 is shared with Python python:2.7.16 and python:2 but previously it was probably shared with python:2.7.15, python:2.7.14 etc. (in other words, python:2.7 is following python:2.7.x as it upgrades)

Your machine and live server probably pulled the image at a different time and now have a different image tagged 2.7. The "shared" tags seems to be like latest tags and may point to newer images as they are released.

What you can do:

• Enforce image pull when building even if image is already present (using docker build with --pull option
• Use a documented Simple tag instead, these should be more consistent (such as python:2.7.16-alpine3.9)
• Do not re-build images during your release process, only build once and use the same image in your local and live environment (see below)

---

EDIT: this can be put into evidence with:

docker images --filter "reference=python" --digests --format "{{.Digest}} {{.Repository}}:{{.Tag}}"
sha256:7a61a96567a2b2ba5db636c83ffa18db584da4024fa5839665e330934cb6b2b2 python:2
sha256:7a61a96567a2b2ba5db636c83ffa18db584da4024fa5839665e330934cb6b2b2 python:2.7
sha256:7a61a96567a2b2ba5db636c83ffa18db584da4024fa5839665e330934cb6b2b2 python:2.7.16
sha256:39224960015b9c0fce12e08037692e8a4be2e940e73a36ed0c86332ce5ce325b python:2.7.15

To precise on:

I thought the point of Docker is that everything runs using the same image and that you shouldn't run into issues like this.

Why is this the case and what can I do to fix it from here so that it runs the same on both dev and live?

Yes, and the recommended pattern is build image once and use that same image trough all your release process - this ensure you have the exact same context (packages, code, etc.) from development to production. You should not re-build your image from scratch on your live server, but ideally build it during your development phase and use that same image for testing and deploying.