Docker missing layer IDs in output

Question

I just did a fresh install of Docker on Ubuntu using the official guidelines: https://docs.docker.com/engine/installation/linux/ubuntulinux/

When I pull an image using "sudo docker pull ubuntu" and then "sudo docker history ubuntu" it returns missing layer IDs in the columns. Using the documentation example ( https://docs.docker.com/engine/reference/commandline/history/ ) my output is:

IMAGE CREATED CREATED BY SIZE COMMENT 3e23a5875458 8 days ago /bin/sh -c #(nop) ENV LC_ALL=C.UTF-8 0 B "missing" 8 days ago /bin/sh -c dpkg-reconfigure locales && loc 1.245 MB "missing" 8 days ago /bin/sh -c apt-get update && apt-get install 338.3 MB 

and so on. Only the base layer ID shows, the rest is "missing". I tried installing on another Ubuntu machine connected to a different network and it has the same problem for any image that I download.

Does someone know what's causing this or able to help me fix it? I rely on this layer ID since I'm gathering some statistics on the reusability of layers so I need this ID to show correctly.

Answer 1

As mentioned in your issue 20131, this could be the consequence of the new docker 1.10 content addressability migration

From the Docker blog post:

Starting from v1.10 we completely change the way Docker addresses the image data on disk.
Previously, every image and layer used a randomly assigned UUID.
In 1.10 we implemented a content addressable method using an ID, based on a secure hash of the image and layer data.

That is why thaJeztah comments:

I think this is expected; the content-addressable storage no longer uses "parent" images to chain the image-layers together.
Newly pulled images also no longer will show the intermediate images (these "missing" images will only be shown for images that were present on the host, but have been migrated to the new storage)

---

Update June 2016 (3 months later)

Nigel Brown has a detailed article about those "missing" images.

Explaining Docker Image IDs

A layer or 'diff' is created during the Docker image build process, and results when commands are run in a container, which produce new or modified files and directories.
These new or modified files and directories are 'committed' as a new layer.

Historically (pre Docker v1.10), each time a new layer was created as a result of a commit action, Docker also created a corresponding image, which was identified by a randomly generated 256-bit UUID, usually referred to as an image ID

One of the big drivers for change, came from the lack of a means of detecting whether an image's contents had been tampered with during a push to or pull from a registry, such as the Docker Hub. This led to robust criticism from the community at large, and led to a series of changes, culminating in content addressable IDs.

Since Docker v1.10, generally, images and layers are no longer synonymous.
Instead, an image directly references one or more layers that eventually contribute to a derived container's filesystem.

Layers are now identified by a digest, which takes the form algorithm:hex;

A Docker image now consists of a configuration object, which (amongst other things) contains an ordered list of layer digests, which enables the Docker Engine to assemble a container's filesystem with reference to layer digests rather than parent images.

So, when a Docker image is pulled from a registry, and the docker history command is used to reveal its contents, the output provides something similar to:

$ docker history swarm IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT   c54bba046158        9 days ago          /bin/sh -c #(nop) CMD ["--help"]                0 B   <missing>           9 days ago          /bin/sh -c #(nop) ENTRYPOINT &{["/swarm"]}      0 B   <missing>           9 days ago          /bin/sh -c #(nop) VOLUME [/.swarm]              0 B   <missing>           9 days ago          /bin/sh -c #(nop) EXPOSE 2375/tcp               0 B   <missing>           9 days ago          /bin/sh -c #(nop) ENV SWARM_HOST=:2375          0 B   <missing>           9 days ago          /bin/sh -c #(nop) COPY dir:b76b2255a3b423981a   0 B   <missing>           9 days ago          /bin/sh -c #(nop) COPY file:5acf949e76228329d   277.2 kB   <missing>           9 days ago          /bin/sh -c #(nop) COPY file:a2157cec2320f541a   19.06 MB   

The <missing> value in the IMAGE field for all but one of the layers of the image, is misleading and a little unfortunate. It conveys the suggestion of an error, but there is no error as layers are no longer synonymous with a corresponding image and ID.
I think it would have been more appropriate to have left the field blank.

Also, the image ID appears to be associated with the uppermost layer, but in fact, the image ID doesn't 'belong' to any of the layers. Rather, the layers collectively belong to the image, and provide its filesystem definition.

But (local vs. remote images):

locally built images on a Docker host are treated slightly differently.
The generic content of an image built locally remains the same - it is a configuration object containing configuration items, including an ordered list of layer digests.

However, when a layer is committed during an image build on a local Docker host, an 'intermediate' image is created at the same time.
Just like all other images, it has a configuration item which is a list of the layer digests that are to be incorporated as part of the image, and its ID or digest contains a hash of the configuration object. Intermediate images aren't tagged with a name, but, they do have a 'Parent' key, which contains the ID of the parent image.

The purpose of the intermediate images and the reference to parent images, is to facilitate the use of Docker's build cache.

$ docker history jbloggs/my_image:latest  IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT   26cca5b0c787        52 seconds ago      /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/bin/b   0 B   97e47fb9e0a6        52 seconds ago      /bin/sh -c apt-get update &&     apt-get inst   16.98 MB   1742affe03b5        13 days ago         /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B   <missing>           13 days ago         /bin/sh -c #(nop) ADD file:5d8521419ad6cfb695   125.1 MB   

In this example, the top two layers are created during the local image build, whilst the bottom layers came from the base image for the build (e.g. Dockerfile instruction FROM debian).

We can use the docker inspect command to review the layer digests associated with the image:

The docker history command shows the image as having four layers, but docker inspect suggests just three layers.
This is because the two CMD instructions produce metadata for the image, don't add any content, and therefore the 'diff' is empty.
The digest 5f70bf18a08a is the SHA256 hash of an empty layer, and is shared by both of the layers in question.

When a locally built image is pushed to a registry, it is only the leaf image that is uploaded along with its constituent layers, and a subsequent pull by another Docker host will not yield any intermediate parent images.

This is because once the image is made available to other potential users on different Docker hosts via a registry, it effectively becomes read-only, and the components that support the build cache are no longer required.
Instead of the image ID, <missing> is inserted into the history output in its place.

Finally:

The digests that Docker uses for layer 'diffs' on a Docker host, contain the sha256 hash of the tar archived content of the diff.
Before the layer is uploaded to a registry as part of a push, it is compressed for bandwidth efficiency. A manifest is also created to describe the contents of the image, and it contains the digests of the compressed layer content. Consequently, the digests for the layers in the manifest are different to those generated in their uncompressed state.