How to remove intermediate images from a build after the build?

Question

When you build your multi-stage Dockerfile with

docker build -t myimage .

it produces the final image tagged myimage, and also intermediate images. To be completely clear we are talking here not about containers, but about images. It looks like this:

See these <none> images? These are what I'm talking about.

Now this "issue" has been discussed to some extent here and here.

Here are some relevant parts:

If these intermediate images would be purged/pruned automatically, the build cache would be gone with each build, therefore forcing you to rebuild the entire image each time.

So okay, it does not make sense to prune then automatically.

Some people do this:

For now, I'm using docker image prune -f after my docker build -t app . command to cleanup those intermediate images.

But unfortunately this is not something I can do. As one discussion participant commented:

It removes "all dangling images", so in shared environments (like Jenkins slave) it's more akin to shooting oneself in the foot. :)

And this is a scenario I found myself in.

So nothing to be "fixed" on Docker side. But how can I remove those extra images, from a single particular build only?

Update

After reading very nice answer from d4nyll below, which is a big step forward, I'd like to add some more constraints to the question ;) First, let me sum up the answer:

• One can use ARG to pass a build id from CI/CD to Dockerfile builder
• Then one can use LABEL syntax to add build id metadata to the stage images being built
• Then one can use the --filter option of docker image prune command to remove only the images with the current build id

This is a big step forward, but I'm still struggling into how to fit it into my usage scenario without adding unnecessary complexity.

In my case a requirement is that application developers who author the Dockerfiles and check them into the source control system are responsible for making sure that their Dockerfiles build the image to their satisfaction. They are not required to craft all their Dockerfiles in a specific way, "so our CI/CD process does not break". They simply have to provide a Dockerfile that produce correct docker image.

Thus, I'm not really in a position to request them to add stuff in the Dockerfile for every single application, just for the sake of CI/CD pipeline. This is something that CI/CD pipeline is expected to handle all by itself.

The only way I can see making this work is to write a Dockerfile parser, that will detect multi-staged build and inject a label per stage and then build that modified Dockerfile. This is a complexity that I'm very hesitant to add to the CI/CD pipeline.

Do I have a better (read simpler) options?

Answer 1

As ZachEddy and thaJeztah mentioned in one of the issues you linked to, you can label the intermediate images and docker image prune those images based on this label.

Dockerfile (using multi-stage builds)

FROM node as builder
LABEL stage=builder
...

FROM node:dubnium-alpine
...

After you've built you image, run:

$ docker image prune --filter label=stage=builder

---

For Automation Servers (e.g. Jenkins)

If you are running the builds in an automation server (e.g. Jenkins), and want to remove only the intermediate images from that build, you can

1. Set a unique build ID as an environment variable inside your Jenkins build
2. Add an ARG instruction for this build ID inside your Dockerfile
3. Pass the build ID to docker build through the --build-arg flag

FROM node as builder
ARG BUILD_ID
LABEL stage=builder
LABEL build=$BUILD_ID
...

FROM node:dubnium-alpine
...

$ docker build --build-arg BUILD_ID .
$ docker image prune --filter label=stage=builder --filter label=build=$BUILD_ID

If you want to persists the build ID in the image (perhaps as a form of documentation accessible within the container), you can add another ENV instruction that takes the value of the ARG build argument. This also allows you to use the similar environment replacement to set the label value to the build ID.

FROM node as builder
ARG BUILD_ID
ENV BUILD_ID=$BUILD_ID
LABEL stage=builder
LABEL build=$BUILD_ID
...

FROM node:dubnium-alpine
...

Answer 2

We're doing exactly this, applying labels to the Dockerfile at build-time like this:

sed -i '/^FROM/a\
LABEL build_id=${env.BUILD_TAG}\
' Dockerfile

Probably too late to help the OP, but hopefully this will be useful to someone facing the same problem.