DockerHub: sha digest doesn't match

Question

I want to determine the sha digest for the latest docker image in DockerHub:

if I try to pull the latest image I can see the digest

# docker pull mysql:latest
...
Digest: sha256:c93ba1bafd65888947f5cd8bd45deb7b996885ec2a16c574c530c389335e9169
Status: Downloaded newer image for mysql:latest
docker.io/library/mysql:latest

so the digest is c93ba1

But if I go to https://hub.docker.com I'll see

So there are 2 digests: 511ca265b41c and 9a355d5c4ec0.

Also if I try to pull with any of the specified digests:

# docker pull mysql@sha256:9a355d5c4ec0351a954b11b494c597cd4e6ab2c8a04ce69c5f8332819890c43e
Error response from daemon: manifest for mysql@sha256:9a355d5c4ec0351a954b11b494c597cd4e6ab2c8a04ce69c5f8332819890c43e not found: manifest unknown: manifest unknown

# docker pull mysql@sha256:511ca265b41cabb694fda409b8ae87fb0a83db15cfb8429d581c33c7aafacddf
Error response from daemon: manifest for mysql@sha256:511ca265b41cabb694fda409b8ae87fb0a83db15cfb8429d581c33c7aafacddf not found: manifest unknown: manifest unknown

But if I try with the one that I got from the pull command it works:

docker pull mysql@sha256:c93ba1bafd65888947f5cd8bd45deb7b996885ec2a16c574c530c389335e9169
sha256:c93ba1bafd65888947f5cd8bd45deb7b996885ec2a16c574c530c389335e9169: Pulling from library/mysql
Digest: sha256:c93ba1bafd65888947f5cd8bd45deb7b996885ec2a16c574c530c389335e9169
Status: Image is up to date for mysql@sha256:c93ba1bafd65888947f5cd8bd45deb7b996885ec2a16c574c530c389335e9169
docker.io/library/mysql@sha256:c93ba1bafd65888947f5cd8bd45deb7b996885ec2a16c574c530c389335e9169

What 'm I doing wrong? And is there a way to get the digest without pulling the image?

Update: Repeated the same procedure again on MacOs using Docker Desktop 2.2

docker pull mysql:8
8: Pulling from library/mysql
...
Digest: sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe
Status: Downloaded newer image for mysql:8
docker.io/library/mysql:8

Then on docker hub

Then

docker image inspect mysql:8
[
    {
        "Id": "sha256:c8ad2be69a220e93826a6308458627b8d5624dc981050fabf950e5de5a7a08a8",
        "RepoTags": [
            "mysql:8"
        ],
        "RepoDigests": [
            "mysql@sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe"
        ]

And

docker pull mysql@sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c
sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c: Pulling from library/mysql
Digest: sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c
Status: Downloaded newer image for mysql@sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c
docker.io/library/mysql@sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c

docker pull mysql@sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe
sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe: Pulling from library/mysql
Digest: sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe
Status: Image is up to date for mysql@sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe
docker.io/library/mysql@sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe

And when I do the same on Linux VM:

sudo docker pull mysql:8
8: Pulling from library/mysql
...
Digest: sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe
Status: Downloaded newer image for mysql:8
docker.io/library/mysql:8

So I have no idea what is happening here

Answer 1

According to this github comment, apparently the CLI when calculating the sha256 digest uses a manifest containing all of the different machine architecture options, while each digest on the DockerHub page is calculated using a manifest with only that specific individual architecture.

UPDATE: It's been a while since I tracked this down, so I don't remember the details, but for more info check out this page. I believe it has to do with the difference between the sha for an image manifest, and the sha for the manifest list...

Answer 2

What you see when pulling the image is the digest for the manifest list or OCI index when the image is packaged as a multi-platform image (even when that packaging only has a single platform). To use the example from the question:

$ regctl manifest get mysql@sha256:f91e704ffa9f19b9a267d9321550a0772a1b64902226d739d3527fd6edbe3dfe --format body | jq .
{
  "manifests": [
    {
      "digest": "sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      },
      "size": 2828
    }
  ],
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "schemaVersion": 2
}

That is a docker manifest list that includes a single platform specific manifest (for linux/amd64) with digest sha256:a592539c5a616b6642bb48822688b6917b373a1293638f9268e8da33e5e9dd1c that you see reported on Docker Hub. Docker tracks the manifest list on a pull, but it always dereferences a multi-platform image to a single platform when pulling to the docker engine.

You typically want to use the multi-platform digest when pinning a digest in a deployment. That allows the same digest can be used on different platforms, though in this specific case that's less important. To get that digest, tools like regclient, crane, and skopeo can be used. And buildx also includes a hidden command docker buildx imagetools inspect that can do this.