Docker how to ADD a file without committing it to an image?

Question

I have a ~300Mb zipped local file that I add to a docker image. The next state then extracts the image.

The problem is that the ADD statement results in a commit that results in a new file system layer makes the image ~300Mb larger than it needs to be.

ADD /files/apache-stratos.zip /opt/apache-stratos.zip
RUN unzip -q apache-stratos.zip && \
    rm apache-stratos.zip && \
    mv apache-stratos-* apache-stratos

Question: Is there a work-around to ADD local files without causing a commit?

One option is to run a simple web server (e.g. python -m SimpleHTTPServer) before starting the docker build, and then using wget to retrieve the file, but that seems a bit messy:

RUN wget http://localhost:8000/apache-stratos.zip && \
    unzip -q apache-stratos.zip && \
    rm apache-stratos.zip && \
    mv apache-stratos-* apache-stratos

Another option is to extract the zipped file at container start up instead of build time, but I would prefer to keep the start up as quick as possible.

Answer 1

There are currently 3 options I can think of.

---

Option 1: you can switch to a tar or compressed tar format from the zip file and then allow ADD to decompress the file for you.

ADD /files/apache-stratos.tgz /opt/

Only downside is any other changes, like a directory rename, will trigger the copy on write again, so you need to make sure your tar file has the contents in the final directory structure.

---

Option 2: Use a multi-stage build. Extract the file in an early stage, perform any changes, and then copy the resulting directory to your final stage. This is a good option for any build engines that cannot use BuildKit. augurar's answer covers this so I won't repeat the same Dockerfile he already has.

---

Option 3: BuildKit (available in 18.09 and newer) allows you to mount files from other locations, including your build context, within a RUN command. This currently requires the experimental syntax. The resulting Dockerfile looks like:

# syntax=docker/dockerfile:experimental
FROM ...
...
RUN --mount=type=bind,source=/files/apache-stratos.zip,target=/opt/apache-stratos.zip \
    unzip -q apache-stratos.zip && \
    rm apache-stratos.zip && \
    mv apache-stratos-* apache-stratos

Then to build that, you export a variable before running your build (you could also export it in your .bashrc or equivalent):

DOCKER_BUILDKIT=1 docker build -t your_image .

More details on BuildKit's experimental features are available here: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md

Answer 2

With Docker 17.05+ you can use a multi-stage build to avoid creating extra layers.

FROM ... as stage1
# No need to clean up here, these layers will be discarded
ADD /files/apache-stratos.zip /opt/apache-stratos.zip
RUN unzip -q apache-stratos.zip
    && mv apache-stratos-* apache-stratos

FROM ...
COPY --from=stage1 apache-stratos/ apache-stratos/