Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GitLab CI/CD: building multiarch Docker images

I want an easy way to build multiarch Docker images in a GitLab runner. By easy, I mean that I just would have to add a .gitlab-ci.yml in my project and it would work.

Here is the .gitlab-ci.yml that I wrote. It builds a multiarch image using buildx and then pushes it to the GitLab registry:

image: cl00e9ment/buildx

services:
- name: docker:dind

variables:
  PLATFORMS: linux/amd64,linux/arm64
  TAG: latest

before_script:
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"

build:
  stage: build
  script:
  - docker buildx build --platform "$PLATFORMS" -t "${CI_REGISTRY_IMAGE}:${TAG}" . --push

The problem is that the linux/arm64 platform isn't available.

Here is how I built the cl00e9ment/buildx image (strongly inspired from snadn/docker-buildx):

Here is the Dockerfile:

FROM docker:latest

ENV DOCKER_CLI_EXPERIMENTAL=enabled
ENV DOCKER_HOST=tcp://docker:2375/

RUN mkdir -p ~/.docker/cli-plugins \
  && wget -qO- https://api.github.com/repos/docker/buildx/releases/latest | grep "browser_download_url.*linux-amd64" | cut -d : -f 2,3 | tr -d '"' | xargs wget -O ~/.docker/cli-plugins/docker-buildx \
  && chmod a+x ~/.docker/cli-plugins/docker-buildx
RUN docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
RUN docker context create buildx \
    && docker buildx create buildx --name mybuilder \
    && docker buildx use mybuilder
RUN docker buildx inspect --bootstrap

...add here is the .gitlab-ci.yml file used to build and push the cl00e9ment/buildx image:

image: docker:latest

services:
  - name: docker:dind

before_script:
  - docker login -u cl00e9ment -p "$DOCKER_HUB_TOKEN"

build:
  stage: build
  script:
  - docker build --add-host docker:`grep docker /etc/hosts | awk 'NR==1{print $1}'` --network host -t cl00e9ment/buildx .
  - docker run --add-host docker:`grep docker /etc/hosts | awk 'NR==1{print $1}'` --network host cl00e9ment/buildx docker buildx inspect --bootstrap
  - docker push cl00e9ment/buildx

test:
  stage: test
  script:
  - docker run --add-host docker:`grep docker /etc/hosts | awk 'NR==1{print $1}'` --network host cl00e9ment/buildx docker buildx inspect --bootstrap

So what's happening?

  • At the end of the build, in the Dockerfile, I run docker buildx inspect --bootstrap to list the available platforms. It gives linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6. So it's all good.
  • After that, I run it again (just after the build and just before the push) and it still gives linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6.
  • However, on the test stage, when the image is freshly downloaded from Docker Hub on a clean environment, it gives linux/amd64, linux/386.

Why?

like image 274
Cl00e9ment Avatar asked Apr 25 '20 17:04

Cl00e9ment


People also ask

How do I create a Multiarch docker image?

You can build a multi-arch image by creating the individual images for each architecture, pushing them to Docker Hub, and entering docker manifest to combine them within a tagged manifest list. You can then push the manifest list to Docker Hub.

Can we store docker images in GitLab?

With GitLab, you can add a job to your pipeline to build Docker images, and push them to the built-in container registry. Here is how... Prerequisites: for this to work, you will need a gitlab-runner with docker-in-docker configured, and a working Dockerfile.

What is docker image in GitLab CI?

GitLab CI in conjunction with GitLab Runner can use Docker Engine to test and build any application. Docker is an open-source project that allows you to use predefined images to run applications in independent "containers" that are run within a single Linux instance.


1 Answers

There is a lot of outdated and incorrect information on building multiarch images on GitLab CI unfortunately. The seems to change quite frequently as it's still an experimental feature. But as of the time of this post, this is how I got my multiarch build working on GitLab public runners (armv6, armv6, arm64, amd64):

First, one must build and push a Docker image containing the buildx binary. Here is the Dockerfile I am using for that:

FROM docker:latest
ARG BUILDX_VER=0.4.2
RUN mkdir -p /root/.docker/cli-plugins && \
    wget -qO ~/.docker/cli-plugins/docker-buildx \
    https://github.com/docker/buildx/releases/download/v${BUILDX_VER}/buildx-v${BUILDX_VER}.linux-amd64 && \
    chmod +x /root/.docker/cli-plugins/docker-buildx

The current GitLab runner image does not initialize the binfmt handlers correctly despite running the initialization code: https://gitlab.com/gitlab-org/gitlab-runner/-/blob/523854c8/.gitlab/ci/_common.gitlab-ci.yml#L91

So we have to do it in our pipeline. We refer to the comments in MR 1861 of the GitLab Runner code and add in the following magic sauce to our .gitlab-ci.yml:

before_script:
  - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes 

Then we can run the rest of our pipeline script with docker login, docker buildx build --use, docker buildx build --push ... and so on.

Now the runner is ready to build for multiple architectures. My final .gitlab-ci.yml can be seen here: https://github.com/oofnikj/nuttssh/blob/multiarch/.gitlab-ci.yml

like image 147
Jordan Sokolic Avatar answered Oct 13 '22 18:10

Jordan Sokolic