Does putting ARG at top of Dockerfile prevent layer re-use?

Question

If an ARG that is declared at the top of a Dockerfile gets changed, but its value is only used for a RUN command near the end of the Dockerfile, does Docker rebuild the whole image from scratch or is it able to re-use the intermediate image from right before the relevant RUN command?

To better utilize layering, should I place my ARG declarations at the top of the Dockerfile, or just before the section that uses them?

I guess part of my question is whether or not an ARG directive generates an intermediate layer.

Answer 1

If you change the value of a build argument all of the layers after that ARG line will be invalidated. So I guess you should include it just before you use the ARG.

Just before you need it:

 docker build --build-arg TEST_ARG=test .  Sending build context to Docker daemon 2.048 kB  Step 1 : FROM ubuntu  ---> 104bec311bcd  Step 2 : RUN echo "no arg used"  ---> Using cache  ---> 5c29cb363a27  Step 3 : ARG TEST_ARG  ---> Using cache  ---> 73b6080f973b  Step 4 : RUN echo $TEST_ARG  ---> 0acd55c24441  Successfully built 0acd55c24441 

At the top:

docker build --build-arg TEST_ARG=test . Sending build context to Docker daemon 2.048 kB Step 1 : FROM ubuntu  ---> 104bec311bcd Step 2 : ARG TEST_ARG ---> Using cache ---> b611a1023fe3 Step 3 : RUN echo "no arg used" ---> Running in 63e0f803c6b2 no arg used ---> 592311ccad72 Removing intermediate container 63e0f803c6b2 Step 4 : RUN echo $TEST_ARG ---> Running in 1515aa8702f0 test ---> fc2d850fbbeb Removing intermediate container 1515aa8702f0 Successfully built fc2d850fbbeb 

In the first example two layers are used from the cache and in the second one only one layer (funnily enough, the ARG layer itself) is used from the cache.

Answer 2

To be more precise than the accepted response, not all lines are cache invalidated after an ARG declaration. Only those that use ARG values and RUNs. The docker documentation the details:

Impact on build caching

ARG variables are not persisted into the built image as ENV variables are. However, ARG variables do impact the build cache in similar ways. If a Dockerfile defines an ARG variable whose value is different from a previous build, then a “cache miss” occurs upon its first usage, not its definition. In particular, all RUN instructions following an ARG instruction use the ARG variable implicitly (as an environment variable), thus can cause a cache miss. All predefined ARG variables are exempt from caching unless there is a matching ARG statement in the Dockerfile.

• https://docs.docker.com/engine/reference/builder/#impact-on-build-caching

You'll have to move your ARGs under the RUNs that would not need the argument in order to keep layer cache optimized.

For more info:

• https://github.com/moby/moby/issues/18017
• https://github.com/moby/moby/pull/18161
• RUN explanation here: https://github.com/moby/moby/pull/21885