Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to cache package manager downloads for docker builds?

If I run composer install from my host, I hit my local composer cache:

  - Installing deft/iso3166-utility (1.0.0)
    Loading from cache

Yet when building a container having in its Dockerfile:

RUN composer install -n -o --no-dev

I download all the things, e.g.:

  - Installing deft/iso3166-utility (1.0.0)
    Downloading: 100%         

It's expected, yet I like to avoid it. As even on a rebuilt, it would also download everything again.

I would like to have a universal cache for composer that I could also reshare for other docker projects.

I looked into this and found the approach to define a volume in the Dockerfile:

ENV COMPOSER_HOME=/var/composer
VOLUME /var/composer

I added that to my Dockerfile, and expected to only download the files once, and hit the cache afterwards.

Yet when I modify my composer, e.g. remove the -o flag, and rerun docker build ., I expected to hit the cache on build, yet I still download the vendors again.

How are volumes supposed to work to have a data cache inside a docker container?

like image 714
k0pernikus Avatar asked Aug 18 '16 16:08

k0pernikus


People also ask

Where is docker build cache stored?

In a default install, these are located in /var/lib/docker. During a new build, all of these file structures have to be created and written to disk — this is where Docker stores base images.

Where are docker downloads stored?

The docker images, they are stored inside the docker directory: /var/lib/docker/ images are stored there.

Is there a docker cache?

Docker's build-cache is a handy feature. It speeds up Docker builds due to reusing previously created layers. You can use the --no-cache option to disable caching or use a custom Docker build argument to enforce rebuilding from a certain step.

How does docker build cache work?

About the Docker Build Cache Docker images are built in layers, where each layer is an instruction from a Dockerfile. Layers stack on top of each other, adding functionality incrementally. The build process knew the Dockerfile didn't change, so it used the cache from the last build for all four layers.


1 Answers

Use the experimental feature : Docker buildkit (Supported Since docker 18.09, docker-compose 1.25.4)

In your dockerfile

# syntax=docker/dockerfile:experimental
FROM ....
# ......  
RUN --mount=type=cache,target=/var/composer composer install -n -o --no-dev

Now before building, make sure the env var is exported:

export DOCKER_BUILDKIT=1
docker build ....

If you are using docker-compose, make sure to export also COMPOSE_DOCKER_CLI_BUILD :

export COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1
docker-compose build ...

If it does not work with docker-compose, make sure your docker-compose version is above 1.25.4

docker-compose version
like image 79
Abdennour TOUMI Avatar answered Sep 17 '22 13:09

Abdennour TOUMI