How to determine the Docker image ID for a tag via Docker Hub API?

Question

Given a tag `latest`, we want to find out another tag with the same image ID on Docker Hub.

Here is how to find out all tags for a repo with the Docker Hub API v2:

TOKEN=$(curl -s -H "Content-Type: application/json" -X POST -d '{"username": "'${UNAME}'", "password": "'${UPASS}'"}' https://hub.docker.com/v2/users/login/ | jq -r .token)
curl -s -H "Authorization: JWT ${TOKEN}" https://hub.docker.com/v2/repositories/fluent/fluentd/tags/?page_size=100 | jq

(See gist.github.com/kizbitz)

Unfortunately, it doesn't contain the image ID but always a `null` value for this key:

$ curl -s -H "Authorization: JWT ${TOKEN}" https://hub.docker.com/v2/repositories/fluent/fluentd/tags/?page_size=100 | jq
{
  "count": 36,
  "next": null,
  "previous": null,
  "results": [
...
    {
      "name": "v0.14.11",
      "full_size": 11964464,
      "id": 7084687,
      "repository": 219785,
      "creator": 2923,
      "last_updater": 2923,
      "last_updated": "2016-12-27T07:16:41.294807Z",
      "image_id": null,
      "v2": true,
      "platforms": [
        5
      ]
    },
...

Unfortunately, the image ID is something different than the `id` in the JSON above.

$ docker images | grep fluent
docker.io/fluent/fluentd                  v0.14.11            1441d57beff9        3 weeks ago         38.25 MB

Theoretically, it should be possible to access the Docker Manifests and along with the the image ID with this Docker Registry call but it doesn't help either:

$ curl -s -H "Authorization: JWT ${TOKEN}" "https://registry.hub.docker.com/v2/fluent/fluentd/manifests/latest"
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Name":"fluent/fluentd","Action":"pull"}]}]}

(See stackoverflow.com)

Here is a similar issue in the Docker GitHub repo but I still cannot figure out the solution: https://github.com/docker/distribution/issues/1490 .

P.S.: Here is my Docker version with which I tried to push a test image:

$ docker version
Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-5.git037a2f5.fc25.x86_64
 Go version:      go1.7.4
 Git commit:      037a2f5/1.12.6
 Built:           Wed Jan 18 12:11:29 2017
 OS/Arch:         linux/amd64

Answer 1

Docker Registry API v2 uses image digest instead of image ID to distinguish image identity.

The image digest can be obtained from Docker-Content-Digest of the HTTP response header by making the following API call:

$ REPOSITORY=fluent/fluentd

$ TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$REPOSITORY:pull" | jq -r .token)

$ curl -s -D - -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" https://index.docker.io/v2/$REPOSITORY/manifests/latest
HTTP/1.1 200 OK
Content-Length: 1982
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Docker-Content-Digest: sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db
Docker-Distribution-Api-Version: registry/2.0
Etag: "sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db"
Date: Tue, 24 Jan 2017 13:34:53 GMT
Strict-Transport-Security: max-age=31536000
...

All tags can be obtained with the following API call:

$ curl -s -H "Authorization: Bearer $TOKEN" https://index.docker.io/v2/$REPOSITORY/tags/list
{"name":"fluent/fluentd","tags":["edge-onbuild","edge","jemalloc","latest-onbuild","latest","onbuild","stable-onbuild","stable","ubuntu-base","v0.12-latest-onbuild","v0.12-latest","v0.12-onbuild","v0.12.16","v0.12.18","v0.12.19","v0.12.20","v0.12.21","v0.12.23","v0.12.24","v0.12.26-2","v0.12.26-onbuild","v0.12.26","v0.12.27-onbuild","v0.12.27","v0.12.28-onbuild","v0.12.28","v0.12.29-onbuild","v0.12.29","v0.12.30-onbuild","v0.12.30","v0.12.31-onbuild","v0.12.31","v0.12","v0.14-latest-onbuild","v0.14-latest","v0.14-onbuild","v0.14.1","v0.14.10-onbuild","v0.14.10","v0.14.11-onbuild","v0.14.11","v0.14.2","v0.14.6","v0.14.8","v0.14"]}

Based on the above, to find the same digest as a specific tag, it will be a script like the following.

#!/bin/bash

REPOSITORY=$1
TARGET_TAG=$2

# get authorization token
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$REPOSITORY:pull" | jq -r .token)

# find all tags
ALL_TAGS=$(curl -s -H "Authorization: Bearer $TOKEN" https://index.docker.io/v2/$REPOSITORY/tags/list | jq -r .tags[])

# get image digest for target
TARGET_DIGEST=$(curl -s -D - -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" https://index.docker.io/v2/$REPOSITORY/manifests/$TARGET_TAG | grep Docker-Content-Digest | cut -d ' ' -f 2)

# for each tags
for tag in ${ALL_TAGS[@]}; do
  # get image digest
  digest=$(curl -s -D - -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" https://index.docker.io/v2/$REPOSITORY/manifests/$tag | grep Docker-Content-Digest | cut -d ' ' -f 2)

  # check digest
  if [[ $TARGET_DIGEST = $digest ]]; then
    echo "$tag $digest"
  fi
done

The result is as follows:

$ ./find_same_digest.sh fluent/fluentd latest
latest sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db
stable sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db
v0.12.31 sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db
v0.12 sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db

If you want to check the digest of the local image, you can get it with docker images --digests:

$ docker images --digests | grep fluentd
fluent/fluentd                  latest              sha256:eaea1edffc34cff3b5e31ee738ea56e46326f90731b4139a19948814a4f0a4db   1788ee7dcfcc        14 hours ago        35.41 MB