Why digests are different depend on registry?

Question

AFAIK, image digest is a hash of image's manifest body.

When I pull busybox image from docker hub, and push it to my private registry, the digests get different.

$ docker pull busybox
...
Digest: sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4
Status: Downloaded newer image for busybox:latest

$ docker tag busybox myregistry/busybox
$ docker push myregistry/busybox
...
08c2295a7fa5: Pushed
latest: digest: sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac size: 527

$ docker images --digests
myregistry/busybox    latest      sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac   efe10ee6727f        2 weeks ago         1.13MB
busybox               latest      sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4   efe10ee6727f        2 weeks ago         1.13MB

The images are not changed at all, and the image ids are same as each other.

But why image digests get different?

---

Updated:

Interestingly, the digest from another private registry is exactly same with the digest by my private registry.

$ docker image inspect efe10ee6727f
...
"RepoDigests": [
            "myregistry/busybox@sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac",
            "busybox@sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4",
            "anotherregistry/busybox@sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac"
        ],

Answer 1

The digests you are looking at are registry digests, which are different from the image id digest. You can have an image id that has different registry references (and possibly digests) for all the places it has been pushed. You can see the two id's in the inspect output:

$ docker inspect busybox --format 'Id: {{.Id}}
Repo Digest: {{index .RepoDigests 0}}'
Id: sha256:efe10ee6727fe52d2db2eb5045518fe98d8e31fdad1cbdd5e1f737018c349ebb
Repo Digest: busybox@sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4

If the registry is using an old v1 manifest, the repository name and tag are part of that manifest, which means it will change as it's moved between registries:

{
   "name": <name>,
   "tag": <tag>,
   "fsLayers": [
      {
         "blobSum": "<digest>"
      },
      ...
    ]
   ],
   "history": <v1 images>,
   "signature": <JWS>
}

However for OCI manifests and Docker's v2 manifests, this is no longer the case and you should see the same registry digest for the same image:

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "size": 7023,
        "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7"
    },
    "layers": [
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 32654,
            "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 16724,
            "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 73109,
            "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736"
        }
    ]
}

Digests themselves are a sha256 digest of the content, which you can also find in OCI's implementation. When you pull an image locally, some things change, including the layers being decompressed, and multi-platform images are dereferenced to your local platform. Because of those changes, the digest on the content will change and the image ID will not match the registry digest.

Therefore, to compare images between registries, make sure you specify you want a v2 schema with the accept header, otherwise the registry will convert the result back to a v1 schema. In curl, passing those headers looks like:

curl \
  -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
  -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" \
  http://$registry/v2/$repo/manifests/$tag