Push multiple docker images to manifest without tags

Question

I have a GitLab CI job that builds multiple architectures of the same application into docker images. I'd like to push them up to DockerHub without needing to tag each image. That way I can add them to a manifest without polluting the tags directory with a bunch of duplicate images.

For example -- from this set of tags:

• r1234 (a manifest containing the below)
• r1234-amd64
• r1234-arm64
• r1234-armv7
• r1234-gui (a manifest containing the below, each image based on the one above)
• r1234-amd64-gui
• r1234-arm64-gui
• r1234-armv7-gui

to this:

• r1234 (manifest containing the same images, without tags)
• r1234-gui (similarly)

Is this something possible? I made this code right now to push up the tags and make a manifest, but I'm not sure how to adapt it.

- >
        for arch in $DEPLOY_ARCHS; do
          NEW_IMAGE_NAME="${REPO_PATH}:${REVISION}-${arch}${TAG_EXTRAS}"
          docker pull "${INDEV_IMAGE_NAME}-${arch}${TAG_EXTRAS}"
          docker tag "${INDEV_IMAGE_NAME}-${arch}${TAG_EXTRAS}" "${NEW_IMAGE_NAME}"
          docker push "${NEW_IMAGE_NAME}" # I want to be able to push to the manifest without tagging
          export IMAGES="$IMAGES ${NEW_IMAGE_NAME}"
        done
    - docker manifest create ${REPO_PATH}:${REVISION}${TAG_EXTRAS} ${IMAGES}
    - docker manifest push --purge ${REPO_PATH}:${REVISION}${TAG_EXTRAS}

Is this something possible?

Answer 1

You want to build multi-arch images and it can be done using buildx (assuming they share the same dockerfile in the current directory):

docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t yourorg/your-image-name:r1234 --push .

On Gitlab-CI this should look like this :

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

services:
  - docker:dind

build-multi-arch-images:
  stage: build-images
  image: jdrouet/docker-with-buildx:stable
  script:
   - docker buildx create --use
   - docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t yourorg/your-image-name:r1234 --push .

stages:
  - build-images

Then for r1234-gui I would suggest a second dockerfile using this trick at the top:

ARG MYAPP_IMAGE=yourorg/your-image-name:latest
FROM $MYAPP_IMAGE

Which will allow you to provide MYAPP_IMAGE as a command line paramether:

docker buildx build -f yourdockerfile --platform linux/amd64,linux/arm64,linux/arm/v7 -t yourorg/your-image-name:r1234-gui --build-arg MYAPP_IMAGE=yourorg/your-image-name:r1234 --push .

Answer 2

Here's the solution I came to -- though for most people the added steps and workarounds here are unnecessary when compared to Jean's answer. You should go with that answer unless you specifically need what mine provides.

I wanted to run the builds on different machines -- an arm box and an amd64 box -- to achieve better build speeds than qemu could provide. However, GitLab / GitHub runners can't communicate with each other, so I couldn't use buildx's ability to run on multiple machines natively. What I came up with was to build the images, push them to an intermediate registry (or export & import them locally if no registry is avaliable was available), pull & use them as cache for another final buildx.

Something like this in GitLab CI:

# Build your images separately however you need, tagging with -${arch}

# Do testing on the image to make sure it validates, then...

deploy_image:
  stage: deploy
  script:
    - >
      for arch in "amd64 arm64 armv7"; do
        export name="$<YOUR CONTAINER URL AND TAG>-${arch}"
        docker pull "NAME" # or `docker load`
        export ARCH_IMAGES_CACHE="$ARCH_IMAGES_CACHE --cache-from $NAME"
      done
    # Docker will detect that the images we pulled are entirely cacheable, and use those without building
    - docker buildx build --push --pull $ARCH_IMAGES_CACHE --build-arg BUILDKIT_INLINE_CACHE=1 --platform="linux/amd64,linux/arm64,linux/arm/v7" -t "$<YOUR CONTAINER URL>:${CI_COMMIT_SHORT_SHA}" .

There are probably better methods or abstractions that could be done using docker manifest or docker buildx imagetools, but this was the best solution I came up with.