Bust cache bust within Dockerfile without providing external build args

Question

We have a Dockerfile, where at a certain point would like no caching to happen.

Currently we are using

ENV CACHE_BUST=$($RANDOM)

Upon further inspection, funny enough that gets cached:

Step 1/1 : ENV CACHE_BUST=$($RANDOM) ---> Using cache

Is there any way from inside the Dockerfile to bust the cache without passing in a unique build-arg (Like docker build . --build-arg CACHE_BUST=$(date +%s)) in the build step?

Answer 1

Update: Reviewing this one, it looks like you injected the cache busting option incorrectly in two ways:

1. ENV is not an ARG
2. The $(x) syntax is not a variable expansion, you need curly brackets (${}), not parenthesis ($()).

To break the cache on the next run line, the syntax is:

ARG CACHE_BUST
RUN echo "command with external dependencies"

And then build with:

docker build --build-arg CACHE_BUST=$(date +%s) .

Why does that work? Because during the build, the values for ARG are injected into RUN commands as environment variables. Changing an environment variable results in a cache miss on the new build.

---

To bust the cache, one of the inputs needs to change. If the command being run is the same, the cache will be reused even if the command has external dependencies that have changed, since docker cannot see those external dependencies.

Options to work around this include:

1. Passing a build arg that changes (e.g. setting it to a date stamp).
2. Changing a file that gets included into the image with COPY or ADD.
3. Running your build with the --no-cache option.

Since you do not want to do option 1, there is a way to do option 3 on a specific line, but only if you can split up your Dockerfile into 2 parts. The first Dockerfile has all the lines as you have today up to the point you want to break the cache. Then the second Dockerfile has a FROM line to depend on the first Dockerfile, and you build that with the --no-cache option. E.g.

Dockerfile1:

FROM base
RUN normal steps

Dockerfile2

FROM intermediate
RUN curl external.jar>file.jar
RUN other lines that cannot be cached
CMD your cmd

Then build with:

docker build -f Dockerfile1 -t intermediate .
docker build -f Dockerfile2 -t final --no-cache .

The only other option I can think of is to make a new frontend with BuildKit that allows you to inject an explicit cache break, or unique variable that results in a cache break.