Build and push a multi arch docker image

Question

I'm trying to make a customized image based on the official postgres images that work on both Raspbian and Mac OS X/Debian.

When I do a

from postgres:10.4 and build the Dockerfile, it seems that docker implicitly picks the x64 version of postgres and then only builds a new image based on that architecture.

As a result, when I push the image to my GitLab registry and download it again, it doesn't work. It throws an standard_init_linux.go:190: exec user process caused "exec format error", which I assume is another way of saying wrong architecture.

From what I can gather at https://blog.docker.com/2017/09/docker-official-images-now-multi-platform/, supporting multiple architectures in one image makes it a "multi arch". However, despite the many resources, it's not obvious to me how I would go about building my own multi arch image and what the requirements are.

Ideally, I would like to seamlessly be able to build and push a armv7 and x64 version - from both a Raspberry Pi/Raspbian host a Mac/Debian host.

I found this question: Cross-compile multi-arch containers

One of the answers suggest a process. However, the process won't support dockerfiles using the RUN command, which is a requirement for my project.

Answer 1

You would need to build each image for the specific architecture, and then generate a manifest that is a list of each of those images. The commands to do the second part are currently experimental. The documentation shows this example (though most tend to put the architecture in the tag name, e.g. myrepo/myapp:arm-linux-v1):

$ docker manifest create 45.55.81.106:5000/coolapp:v1 \
    45.55.81.106:5000/coolapp-ppc64le-linux:v1 \
    45.55.81.106:5000/coolapp-arm-linux:v1 \
    45.55.81.106:5000/coolapp-amd64-linux:v1 \
    45.55.81.106:5000/coolapp-amd64-windows:v1
Created manifest list 45.55.81.106:5000/coolapp:v1

$ docker manifest annotate 45.55.81.106:5000/coolapp:v1 45.55.81.106:5000/coolapp-arm-linux --arch arm

$ docker manifest push 45.55.81.106:5000/coolapp:v1
Pushed manifest 45.55.81.106:5000/coolapp@sha256:9701edc932223a66e49dd6c894a11db8c2cf4eccd1414f1ec105a623bf16b426 with digest: sha256:f67dcc5fc786f04f0743abfe0ee5dae9bd8caf8efa6c8144f7f2a43889dc513b
Pushed manifest 45.55.81.106:5000/coolapp@sha256:f3b3b28a45160805bb16542c9531888519430e9e6d6ffc09d72261b0d26ff74f with digest: sha256:b64ca0b60356a30971f098c92200b1271257f100a55b351e6bbe985638352f3a
Pushed manifest 45.55.81.106:5000/coolapp@sha256:39dc41c658cf25f33681a41310372f02728925a54aac3598310bfb1770615fc9 with digest: sha256:df436846483aff62bad830b730a0d3b77731bcf98ba5e470a8bbb8e9e346e4e8
Pushed manifest 45.55.81.106:5000/coolapp@sha256:f91b1145cd4ac800b28122313ae9e88ac340bb3f1e3a4cd3e59a3648650f3275 with digest: sha256:5bb8e50aa2edd408bdf3ddf61efb7338ff34a07b762992c9432f1c02fc0e5e62
sha256:050b213d49d7673ba35014f21454c573dcbec75254a08f4a7c34f66a47c06aba

To configure the docker client with experimental features, you can set the following in your local $HOME/.docker/config.json:

{
   "auths": { ... },
   "experimental": "enabled"
}

Note in the above, the auths section is likely already configured if you've performed a docker login. You should leave that section untouched other than to add a comma after the closing bracket. If you don't have an auths section, then exclude that line from your file.

---

However, the process won't support dockerfiles using the RUN command, which is a requirement for my project.

You can download images for different architectures, but you won't be able to run those image from the wrong architecture. The RUN command spawns a temporary container during the build process to run the requested command, which simply will not work. The only reasons I know of to pull an image for another architecture are to relay that image to another location where they can be used (private registry, or export+scp), or for building multi-architecture manifests like you see above.

---

The point here is to build for multiple architectures on a single host. I have seen blog posts using QEMU to achieve this, but they all use different base images for each architecture. It is not clear how to do this when the base image is a multi-arch image already.

If you can build an image without any RUN steps, using a cross compiler for any binaries outside of your image, and only performing steps like COPY to add files to your new image, then you can create everything on a single host. When the upstream image is multi-arch, then you need to dig deeper and find a platform specific tag to use for your upstream image. There are several options for this. The difficult option is to inspect the manifest of the multi-arch tag to see the specific digests:

$ docker manifest inspect busybox
{                    
   "schemaVersion": 2,                                                                       
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [                     
      {                  
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 527,
         "digest": "sha256:bbb143159af9eabdf45511fd5aab4fd2475d4c0e7fd4a5e154b98e838488e510",
         "platform": {                                                       
            "architecture": "amd64",
            "os": "linux"                                                                    
         }            
      },                            
      {                  
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 527,
         "digest": "sha256:3d1b11fb001ece2e99556690ce064b07851649582b1f0907649c21e65ba4298f",
         "platform": {
            "architecture": "arm",                                                                                                           
            "os": "linux",                                                                           
            "variant": "v5"
         }                                                                     
      },
...

And then you can use a specific digest tag, like busybox@sha256:3d1b11fb001ece2e99556690ce064b07851649582b1f0907649c21e65ba4298f. To do that more programmatically, you can parse the output with jq:

docker manifest inspect busybox | \
  jq '.manifests | .[] | select(.platform.architecture == "arm64") | .digest'

However, most multi-arch images include tags for individual architectures. So you would select that specific tag instead of the multi-arch tag. For the official images, docker currently has architecture specific repos documented in their official images repo.