Understanding docker layers and future changes

Question

So,

Each Docker image references a list of read-only layers that represent filesystem differences. Layers are stacked on top of each other to form a base for a container’s root filesystem.

and,

Because each container has its own thin writable container layer, and all changes are stored in this container layer, this means that multiple containers can share access to the same underlying image and yet have their own data state.

and also,

Layers of a Docker image are essentially just files generated from running some command. You can view the contents of each layer on the Docker host at /var/lib/docker/aufs/diff.

Now, questions,

• Say I build my docker images layer by layer. A < B < C < D, etc.
• Now if I update my docker image A, would the rest of docker images B, C, D see the changes as well, provided that the changes are not touched by them when building them? E.g., adding /etc/apt/sources.list.d/somethingnew that was never there before.
• If I had built another set of docker images layer by layer. A < X < Y < Z, then the above changes will be reflected in X, Y, Z as well, right?
• Now if however, the future changes to A, is made to the same file that will be changed when building B, C, D, then what would happen? For e.g., let's make it simple that docker images B, C, D each only add pkgB, pkgC, and pkgD in its layer. If I add a pkgA to A after B, C, D are build, what would happen? -- I guess there should be one single version of truth as what packages are in, for a single system, so what would it be for this case?
• What if I'm only upgrading the packages in A? This should be OK right? Would the rest of docker images see the changes as well?

Answer 1

Overall, each image contains its parent image, either as embedded bytes or as a "hard" reference to an image in your local cache if it is already there.

By "parent" I mean the FROM: someimage instruction in your Dockerfile.

I also wrote "hard" since the reference is actually a sha246 digest of the parent image. If any bit of the parent changes, the digest will be different.

There are three main cases here:

1. You start with a clear cache (docker image ls -a shows nothing). If you docker pull ... some image from a public registry, it will have its parent embedded. A docker ps -a should show only one image.

2. If however you already have the parent image in your cache, the docker pull ... won't download the parent again. In that case, the pulled image has a reference to the parent in your cache.

3. If you build locally from a clear cache, docker will download the parent image and produce a child image with a reference to the parent.

It's still the same result at the end. If you replace the parent image with a newer version, the digest won't be the same.

Docker won't let you delete an image if another image references it. When you push your image to a registry, the parent is embedded (I'm skimping over caching behaviour on the registry side here). I think that you can also embed the parent by using docker export and docker import but I haven't tried it. For instance, docker export B, then delete A and B from your docker cache and docker import B should show only one image.

You can get the actual parent relationship using

docker image inspect <image-id> | grep -E "Id|Parent"

combine that with

docker image ls -a --digests

To inspect the relationships.

Some more info.

When you build an image, the following steps occur:

1. The build context is sent to the host with the docker daemon. This is basically all the files in the directory that your Dockerfile is in. That's why it is important to make use of .dockerignore to only send files that are COPY'd in your Dockerfile.
2. The docker daemon creates a temporary container using the FROM instruction in the Dockerfile. This is the imported image, including its own imported images. Then every instruction in the Dockefile is executed inside that container. When a persistent change is made in the temporary container (like COPY'ing), it then saves its state. This effectively adds a layer to the final image.
3. Once finished executing the DOCKERFILE instructions, the temporary container is destroyed. You are left with the final image.

You can see all the layers in the image using docker history <image-id>

Note that this offers a handy way to debug a Dockerfile. You should see an id for the layers that correspond to persistent instructions in your Dockerfile. You can create a new container from any layer using docker run --rm -it <id next to layer> sh and manually execute the Dockerfile instructions that follow.