docker-compose up can't find module, but running from bash works

Question

I've been building a Node.js app on Docker, and following the advice in John Lees-Miller's excellent Lessons from Building a Node App in Docker for setting up my environment. Everything has been working fine for a week or two, but today I started running into an issue that I cannot figure out.

In a nutshell, I can no longer run the app via docker-compose up; I'm getting a node error for newly added module depedencies:

sfsftp_1  | module.js:327
sfsftp_1  |     throw err;
sfsftp_1  |     ^
sfsftp_1  | 
sfsftp_1  | Error: Cannot find module 'eval'
sfsftp_1  |     at Function.Module._resolveFilename (module.js:325:15)
sfsftp_1  |     at Function.Module._load (module.js:276:25)
sfsftp_1  |     at Module.require (module.js:353:17)
sfsftp_1  |     at require (internal/module.js:12:17)
sfsftp_1  |     at Object.<anonymous> (/home/app/sfsftp/lib/reformatter.js:4:13)
sfsftp_1  |     at Module._compile (module.js:409:26)
sfsftp_1  |     at Object.Module._extensions..js (module.js:416:10)
sfsftp_1  |     at Module.load (module.js:343:32)
sfsftp_1  |     at Function.Module._load (module.js:300:12)
sfsftp_1  |     at Module.require (module.js:353:17)

However, if I run a bash shell:

docker-compose run --rm sfsftp /bin/bash 

and start the node app from inside the container with node sftp.js, the app starts up fine!

I initially had this issue earlier today with the module jsonminify, which I had recently added to the project; I believe I had run the app both ways (docker-compose up and docker-compose run via bash) since making that change, but perhaps not? In the interest of expediency, I decided to punt on the issue, and removed my use on jsonminify. Later, I needed to add the eval module to my project, and now I'm getting the same error for that module. I feel like the docker-compose up command is using an older copy of the image than the docker-compose run command, one without my recent changes.

When I've added the modules, I've followed the process in the article - use docker-compose run to get a bash shell and running npm install --save module inside the running instance. My package.json and npm-shrinkwrap.json files are being updated, and I see the eval module in both.

I'm still fairly new to developing in Docker containers, so I've reached the point of "let's try this"; so far I've tried:

• docker-compose build followed by docker-compose up
• docker-compose up --build
• docker-compose build --no-cache followed by docker-compose up
• docker run -p 22:9001 sftpdocker_sfsftp

The last one worked - running docker directly, instead of via docker-compose. But this loses some of the benefits of the setup. How do I make docker-compose work?

For reference, here is my Dockerfile:

# prepared with reference to http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html
# Current LTS Node version
FROM node:4.6.0

# Let's not run as Root.  And, update to recent NPM
RUN useradd --user-group --create-home --shell /bin/false app &&\
  npm install --global [email protected]

ENV HOME=/home/app

# Get what we need for npm install
COPY package.json npm-shrinkwrap.json $HOME/sfsftp/
RUN chown -R app:app $HOME/*

# and install dependencies
USER app
WORKDIR $HOME/sfsftp
RUN npm install && npm cache clean

# do this after dependencies, so that if only the app changes, npm install won't be rerun
USER root
COPY . $HOME/sfsftp
RUN chown -R app:app $HOME/*
USER app

# Start it up!
CMD ["node", "sftp.js"]

and docker-compose.yml:

sfsftp:
  build: .
  ports:
    - '22:9001'
  volumes:
    - .:/home/app/sfsftp
    - /home/app/sfsftp/node_modules

and finally, package.json:

{
  "name": "sftp2sf",
  "version": "1.0.0",
  "description": "",
  "main": "sftp.js",
  "scripts": {
    "start": "node sftp.js",
    "test": "mocha",
    "coverage": "istanbul cover _mocha -- -R spec && open coverage/lcov-report/index.html"
  },
  "author": "Jason Clark <[email protected]>",
  "license": "UNLICENSED",
  "dependencies": {
    "buffer-equal-constant-time": "^1.0.1",
    "byline": "^5.0.0",
    "eval": "^0.1.1",
    "fast-csv": "^2.3.0",
    "jsforce": "^1.7.0",
    "jsonminify": "^0.4.1",
    "minimatch": "^3.0.3",
    "minimist": "^1.2.0",
    "moment": "^2.15.1",
    "ssh2": "^0.5.2",
    "through": "^2.3.8"
  },
  "devDependencies": {
    "chai": "^3.5.0",
    "istanbul": "^0.4.5",
    "mocha": "^3.1.0",
    "sinon": "^1.17.6",
    "stream-to-array": "^2.3.0"
  }
}

Update: To address a proposed solution, of removing the volumes section of the docker-compose.yml: The volumes section is an important part of this setup, and allows my local host to share files with the running docker image during development, except the node_modules which only live in the image; see the linked article for full explanation. It is worth nothing that I've used this setup with those volume statements since starting on this project, and there are at least a dozen other Node modules that were installed the same way, and they are not causing the exception - if I remove eval, the code works.

Update 17 Nov 2016: This just seemed to fix itself; after switching gears to a higher-priority project for several days, I no longer had the problem. I don't believe in bugs fixing themselves, but I cannot figure out what I may have done to resolve the issue. Today, I had to add a new node module to the project (docker-compose -f docker-compose.yml run --rm sfsftp /bin/bash, npm install --save module from inside the running shell, docker-compose build) and now the problem is back, only it's the new module that cannot be found.

Answer 1

I came across the same issue and found that the solution was to remove the container and then fire it up again, i.e.

npm install some-package --save

docker-compose down

docker-compose up --build

17 Nov 20 - edited the above to make it more succinct

Answer 2

The problem is that the node_modules volume in compose file is set as anonymous, anonymous volumes are preserved when containers are recreated, even with --build, the node still can't find the newly added module. See issues here.

For now, I use these two commands to make it work since the docker-compose v1.9 with --renew-anon-volumes witch I think should be used to handle this situation is not officially released yet:

docker-compose down
docker-compose up --build