Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does putting ARG at top of Dockerfile prevent layer re-use?

If an ARG that is declared at the top of a Dockerfile gets changed, but its value is only used for a RUN command near the end of the Dockerfile, does Docker rebuild the whole image from scratch or is it able to re-use the intermediate image from right before the relevant RUN command?

To better utilize layering, should I place my ARG declarations at the top of the Dockerfile, or just before the section that uses them?

I guess part of my question is whether or not an ARG directive generates an intermediate layer.

like image 940
Woodrow Barlow Avatar asked Jan 11 '17 14:01

Woodrow Barlow


People also ask

What does ARG in Dockerfile do?

You can use the ARG command inside a Dockerfile to define the name of a parameter and its default value. This default value can also be overridden using a simple option with the Docker build command.

Can an ARG variable on Dockerfile be used by the running container?

Running containers can't access values of ARG variables. This also applies to CMD and ENTRYPOINT instructions which just tell what the container should run by default.

What is the difference between ARG and ENV in Dockerfile?

ENV is for future running containers. ARG for building your Docker image. ¶ ENV is mainly meant to provide default values for your future environment variables.

Can I use from twice in Dockerfile?

FROM can appear multiple times within a single Dockerfile in order to create multiple images. Simply make a note of the last image ID output by the commit before each new FROM command.


2 Answers

If you change the value of a build argument all of the layers after that ARG line will be invalidated. So I guess you should include it just before you use the ARG.

Just before you need it:

 docker build --build-arg TEST_ARG=test .  Sending build context to Docker daemon 2.048 kB  Step 1 : FROM ubuntu  ---> 104bec311bcd  Step 2 : RUN echo "no arg used"  ---> Using cache  ---> 5c29cb363a27  Step 3 : ARG TEST_ARG  ---> Using cache  ---> 73b6080f973b  Step 4 : RUN echo $TEST_ARG  ---> 0acd55c24441  Successfully built 0acd55c24441 

At the top:

docker build --build-arg TEST_ARG=test . Sending build context to Docker daemon 2.048 kB Step 1 : FROM ubuntu  ---> 104bec311bcd Step 2 : ARG TEST_ARG ---> Using cache ---> b611a1023fe3 Step 3 : RUN echo "no arg used" ---> Running in 63e0f803c6b2 no arg used ---> 592311ccad72 Removing intermediate container 63e0f803c6b2 Step 4 : RUN echo $TEST_ARG ---> Running in 1515aa8702f0 test ---> fc2d850fbbeb Removing intermediate container 1515aa8702f0 Successfully built fc2d850fbbeb 

In the first example two layers are used from the cache and in the second one only one layer (funnily enough, the ARG layer itself) is used from the cache.

like image 98
eawenden Avatar answered Oct 05 '22 00:10

eawenden


To be more precise than the accepted response, not all lines are cache invalidated after an ARG declaration. Only those that use ARG values and RUNs. The docker documentation the details:

Impact on build caching

ARG variables are not persisted into the built image as ENV variables are. However, ARG variables do impact the build cache in similar ways. If a Dockerfile defines an ARG variable whose value is different from a previous build, then a “cache miss” occurs upon its first usage, not its definition. In particular, all RUN instructions following an ARG instruction use the ARG variable implicitly (as an environment variable), thus can cause a cache miss. All predefined ARG variables are exempt from caching unless there is a matching ARG statement in the Dockerfile.

  • https://docs.docker.com/engine/reference/builder/#impact-on-build-caching

You'll have to move your ARGs under the RUNs that would not need the argument in order to keep layer cache optimized.

For more info:

  • https://github.com/moby/moby/issues/18017
  • https://github.com/moby/moby/pull/18161
  • RUN explanation here: https://github.com/moby/moby/pull/21885
like image 26
Brown nightingale Avatar answered Oct 04 '22 23:10

Brown nightingale