Menu
I am trying to implement Buildkit's cache mount feature.
My Dockerfile is:
# syntax = docker/dockerfile:experimental
FROM python:3.6-alpine
RUN --mount=type=cache,target=/root/.cache/pip pip install pyyaml
CMD: docker build --progress=plain -t abc:1 . --no-cache
First Run Output:
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 175B done
#1 DONE 0.0s
#3 resolve image config for docker.io/docker/dockerfile:experimental
#3 DONE 2.8s
#4 docker-image://docker.io/docker/dockerfile:experimental@sha256:787107d7f...
#4 CACHED
#5 [internal] load metadata for docker.io/library/python:3.6-alpine
#5 DONE 0.0s
#6 [stage-0 1/2] FROM docker.io/library/python:3.6-alpine
#6 CACHED
#7 [stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install...
#7 1.410 Collecting pyyaml
#7 1.731 Downloading PyYAML-5.3.1.tar.gz (269 kB)
#7 2.470 Building wheels for collected packages: pyyaml
#7 2.471 Building wheel for pyyaml (setup.py): started
#7 2.801 Building wheel for pyyaml (setup.py): finished with status 'done'
#7 2.802 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl size=44621 sha256=0bfa8470e677b8c2e1d50f8749aa5890b5fb7d7d15352596fbd928c846f9922d
#7 2.802 Stored in directory: /root/.cache/pip/wheels/e5/9d/ad/2ee53cf262cba1ffd8afe1487eef788ea3f260b7e6232a80fc
#7 2.803 Successfully built pyyaml
#7 2.835 Installing collected packages: pyyaml
#7 2.882 Successfully installed pyyaml-5.3.1
#7 DONE 3.3s
#8 exporting to image
#8 exporting layers
#8 exporting layers 0.1s done
#8 writing image sha256:d7ecd86c64611e14f55fa2d31b8f440246e356026eef7175526ccc921373c205 done
#8 naming to docker.io/library/abc:1 done
#8 DONE 0.1s
Second Run Output:
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 37B done
#2 DONE 0.0s
#3 resolve image config for docker.io/docker/dockerfile:experimental
#3 DONE 2.7s
#4 docker-image://docker.io/docker/dockerfile:experimental@sha256:787107d7f...
#4 CACHED
#5 [internal] load metadata for docker.io/library/python:3.6-alpine
#5 DONE 0.0s
#6 [stage-0 1/2] FROM docker.io/library/python:3.6-alpine
#6 CACHED
#7 [stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install...
#7 1.385 Collecting pyyaml
#7 1.773 Downloading PyYAML-5.3.1.tar.gz (269 kB)
#7 2.517 Building wheels for collected packages: pyyaml
#7 2.518 Building wheel for pyyaml (setup.py): started
#7 2.844 Building wheel for pyyaml (setup.py): finished with status 'done'
#7 2.846 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl size=44621 sha256=064a160d2f195ad7753fba5abf9481b40b515fadffc94f30aef20146209a0faa
#7 2.846 Stored in directory: /root/.cache/pip/wheels/e5/9d/ad/2ee53cf262cba1ffd8afe1487eef788ea3f260b7e6232a80fc
#7 2.846 Successfully built pyyaml
#7 2.877 Installing collected packages: pyyaml
#7 2.920 Successfully installed pyyaml-5.3.1
#7 DONE 3.3s
#8 exporting to image
#8 exporting layers
#8 exporting layers 0.1s done
#8 writing image sha256:42525b426b00a8728b735c15efb96bd237fc020f46d6d99f5bfebb7758016e71 done
#8 naming to docker.io/library/abc:1 done
#8 DONE 0.1s
Expected:
In the second run, I was expecting pyyaml to be picked from the cache.
OS: Mac OS Catalina (10.15.3)
Docker Version:
Ref: https://stackoverflow.com/a/58021389/1060337
301
To enable BuildKit builds Easiest way from a fresh install of docker is to set the DOCKER_BUILDKIT=1 environment variable when invoking the docker build command, such as: $ DOCKER_BUILDKIT=1 docker build .
BuildKit has been lingering in the background of Docker builds for some time now as an experimental feature. Since 19.03, with an environment variable BuildKit can be enabled and unleash some massive performance features.
Docker BuildKit is the next generation container image builder, which helps us to make Docker images more efficient, secure, and faster. It's integrated into the Docker release version v18.06. BuildKit is a part of the Moby project which was developed after learning's and failures to make the image build process -
Enabling BuildKit in your build If you're using Docker Desktop on macOS or Windows: If you've newly installed it since October 2020, or have reset to factory defaults, BuildKit will be enabled by default for all builds. You can turn it on/off for all builds in Preferences > Docker Engine.
You could add mode=0777 for example to Dockerfile to make it work.
Dockerfile:
# syntax = docker/dockerfile:experimental
FROM python:3.6-alpine
RUN --mount=type=cache,mode=0777,target=/root/.cache/pip pip install pyyaml
Here,
mode: File mode for new cache directory in octal. Default 0755.
The 1st run:
#8 [stage-0 2/2] RUN --mount=type=cache,mode=0777,target=/root/.cache/pip p...
#8 2.085 Collecting pyyaml
#8 2.455 Downloading PyYAML-5.3.1.tar.gz (269 kB)
#8 3.484 Building wheels for collected packages: pyyaml
#8 3.485 Building wheel for pyyaml (setup.py): started
#8 3.864 Building wheel for pyyaml (setup.py): finished with status 'done'
#8 3.865 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl size=44621 sha256=372ab164f3e88d66fa2571fa048317b22a7c48bf723dfecbac4e058e9037c4fd
#8 3.865 Stored in directory: /root/.cache/pip/wheels/e5/9d/ad/2ee53cf262cba1ffd8afe1487eef788ea3f260b7e6232a80fc
#8 3.866 Successfully built pyyaml
#8 3.901 Installing collected packages: pyyaml
#8 3.961 Successfully installed pyyaml-5.3.1
#8 DONE 4.7s
The 2nd run:
#8 [stage-0 2/2] RUN --mount=type=cache,mode=0777,target=/root/.cache/pip p...
#8 2.069 Processing /root/.cache/pip/wheels/e5/9d/ad/2ee53cf262cba1ffd8afe1487eef788ea3f260b7e6232a80fc/PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl
#8 2.112 Installing collected packages: pyyaml
#8 2.166 Successfully installed pyyaml-5.3.1
#8 DONE 2.5s
UPDATE:
If I use next for experiment:
RUN --mount=type=cache,target=/root/.cache/pip ls -alh /root/.cache/pip;pip install pyyaml;ls -alh /root/.cache/pip
I could see next:
#7 [stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip ls -alh /ro...
#7 0.405 total 8K
#7 0.405 drwxr-xr-x 2 root root 4.0K Apr 29 01:04 .
#7 0.405 drwxr-xr-x 3 root root 4.0K Apr 29 01:04 ..
#7 1.957 Collecting pyyaml
#7 2.572 Downloading PyYAML-5.3.1.tar.gz (269 kB)
#7 3.552 Building wheels for collected packages: pyyaml
#7 3.553 Building wheel for pyyaml (setup.py): started
#7 3.885 Building wheel for pyyaml (setup.py): finished with status 'done'
#7 3.887 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl size=44621 sha256=4c971f0f901ff8ed3a20c5bb7d94069f15249c665ebe611430db17cb87969cc7
#7 3.887 Stored in directory: /root/.cache/pip/wheels/e5/9d/ad/2ee53cf262cba1ffd8afe1487eef788ea3f260b7e6232a80fc
#7 3.888 Successfully built pyyaml
#7 3.918 Installing collected packages: pyyaml
#7 3.972 Successfully installed pyyaml-5.3.1
#7 4.451 WARNING: You are using pip version 20.0.2; however, version 20.1 is available.
#7 4.451 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
#7 4.512 total 20K
#7 4.512 drwxr-xr-x 5 root root 4.0K Apr 29 01:04 .
#7 4.512 drwxr-xr-x 3 root root 4.0K Apr 29 01:04 ..
#7 4.512 drwxr-xr-x 5 root root 4.0K Apr 29 01:04 http
#7 4.512 drwxr-xr-x 2 root root 4.0K Apr 29 01:04 selfcheck
#7 4.512 drwxr-xr-x 3 root root 4.0K Apr 29 01:04 wheels
#7 DONE 4.7s
Rerun above again, the things same. So looks like if not set mode, the buildkit won't mount a cache from docker host. But if set a mode mode, even with 0755, it's ok.
So, now I strongly guess it's a bug of buildkit on MACOS, that is: if not set mode, it will not mount cache for you. But the behavior is ok on linux.
183
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
