Docker compose missing yarn dependencies on build

Question

Can't get node_modules folder when running yarn install in the Dockerfile

test-sof
├── docker-compose.yml
├── Dockerfile
├── package.json
└── yarn.lock

docker-compose.yml

version: '3'
services:
  web:
    build: .
    volumes:
      - .:/myapp

package.json

{
  "name": "site",
  "private": true,
  "dependencies": {
    "@rails/webpacker": "^3.2.1",
    "babel-preset-react": "^6.24.1",
    "prop-types": "^15.6.0",
    "react": "^16.2.0",
    "react-dom": "^16.2.0",
    "reactjs": "^1.0.0",
    "underscore": "^1.8.3"
  },
  "devDependencies": {
    "webpack-dev-server": "^2.11.1"
  }
}

Dockferfile

FROM ruby:2.5

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash - && \
    curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
    echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
    apt-get update && \
    apt-get install -qq -y build-essential libpq-dev nodejs yarn

RUN mkdir /myapp
WORKDIR /myapp

ADD ./package.json /myapp/

RUN yarn install

output of the step RUN yarn install when docker-compose build:

Step 6/6 : RUN yarn install
---> Running in 3a0e7095ec81 
yarn install v1.3.2 
info No lockfile found. 
[1/4] Resolving packages... 
[2/4] Fetching packages... 
info [email protected]: The platform "linux" is incompatible with this module. 
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation. 
[3/4] Linking dependencies... 
warning "@rails/webpacker > [email protected]" has unmet peer dependency "caniuse-lite@^1.0.30000697". 
warning " > [email protected]" has unmet peer dependency "webpack@^2.2.0 || ^3.0.0". 
warning "webpack-dev-server > [email protected]" has unmet peer dependency "webpack@^1.0.0 || ^2.0.0 || ^3.0.0". 
[4/4] Building fresh packages... 
success Saved lockfile. 
Done in 21.11s. 
Removing intermediate container 3a0e7095ec81  
---> 5720579a0f2a 
Successfully built 5720579a0f2a 
Successfully tagged testsof_web:latest

Running command: docker-compose run web bash to get in the container

root@11af1818e494:/myapp# ls    
Dockerfile  docker-compose.yml  package.json

no node_modules folder present, but later when running inside the container: yarn install output:

root@11af1818e494:/myapp# yarn install
yarn install v1.3.2
info No lockfile found. 
[1/4] Resolving packages... 
[2/4] Fetching packages... 
info [email protected]: The platform "linux" is incompatible with this module. 
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation. 
[3/4] Linking dependencies... 
warning "@rails/webpacker > [email protected]" has unmet peer dependency "caniuse-lite@^1.0.30000697". 
warning " > [email protected]" has unmet peer dependency "webpack@^2.2.0 || ^3.0.0". 
warning "webpack-dev-server > [email protected]" has unmet peer dependency "webpack@^1.0.0 || ^2.0.0 || ^3.0.0". 
[4/4] Building fresh packages... 
success Saved lockfile. 
Done in 13.03s.

then when listing:

root@11af1818e494:/myapp# ls    
Dockerfile  docker-compose.yml  node_modules  package.json  yarn.lock

folder node_modules IT IS present. Why?

Answer 1

This part of Dockerfile installs yarn packages:

RUN mkdir /myapp
WORKDIR /myapp
ADD ./package.json /myapp/
RUN yarn install

Folder /myapp is created, package.json is copied to it and yarn packages are installed. Build is successful and, of course, node_modules folder is inside built image.

But after that you start built image with:

volumes:
  - .:/myapp

which means that content of folder where docker-compose.yaml is is mounted to /myapp folder inside container, so it covers content of container's /myapp folder.

You don't need to mount current folder to container's folder to achieve what you want. Just delete it from your docker-compose.yaml:

version: '3'
services:
  web:
    build: .

Now you can:

$ docker-compose build
$ docker-compose run web bash
root@558d5b0c2ccb:/myapp# ls -la
total 268
drwxr-xr-x   3 root root   4096 Feb 23 22:25 .
drwxr-xr-x  65 root root   4096 Feb 23 22:36 ..
drwxr-xr-x 818 root root  36864 Feb 23 22:25 node_modules
-rw-rw-r--   1 root root    333 Feb 23 22:07 package.json
-rw-r--r--   1 root root 219075 Feb 23 22:25 yarn.lock

EDIT:

But what I want is when building the image, get these dependencies not when spinning up the containers. Otherwise I have another container which mounts de source code and needs this node_modules folder when running the "docker-compose up" and I'd like to avoid some kind of ugly sleep until the node_modules is finished. So I need present this folder on my root host before up the containers somehow

If you want to achieve the above goal, you can use the following workaround:

1. You modify Dockerfile a little:

FROM ruby:2.5

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash - && \
    curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
    echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
    apt-get update && \
    apt-get install -qq -y build-essential libpq-dev nodejs yarn

RUN mkdir /build && mkdir /myapp
WORKDIR /build

ADD ./package.json /build/

RUN yarn install

WORKDIR /myapp

CMD cp -a /build/node_modules/ /myapp/

That's means that yarn packages will be built in /build folder inside image and copied to /myapp folder once container is started.

2. You use the original docker-compose.yaml file:

version: '3'
services:
  web:
    build: .
    volumes:
      - .:/myapp

when you start web container:

docker-compose up web

folder node_modules is copied to mounted folder i.e. to . folder on your host machine.

3. Now you can start any container and it will be contain node_modules folder inside /myapp:

docker-compose run web bash

So, you will be able to achieve your goal the following way:

$ docker-compose build && docker-compose up web
$ docker-compose run web bash
root@4b38e60adfa3:/myapp# ls -la
total 64
drwxrwxr-x   3 1000 1000  4096 Feb 24 10:59 .
drwxr-xr-x  66 root root  4096 Feb 24 11:13 ..
-rw-rw-r--   1 1000 1000   497 Feb 24 10:55 Dockerfile
-rw-rw-r--   1 1000 1000    73 Feb 24 09:02 docker-compose.yaml
drwxr-xr-x 818 root root 40960 Feb 24 10:57 node_modules
-rw-rw-r--   1 root root   333 Feb 23 22:07 package.json