NPM Install failing in docker container - npm WARN tar ENOENT: no such file or directory

Question

I am trying to compile a Nativescript application as part of our Teamcity deployment strategy.

When I run NPM install, I get a ENOENT error trying to find files, as shown below:

  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignAll.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/create.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignAllWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assign.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/curry.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignAll.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignIn.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/curryN.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignAllWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignInAll.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignInAllWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/curryRight.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignIn.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/curryRightN.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignInAll.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignInWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignInAllWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/date.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assignWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assoc.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/debounce.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/assocPath.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignInWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/deburr.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/at.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assignWith.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/defaults.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assoc.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/attempt.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/defaultsAll.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/assocPath.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/browserify-117e4a8d/test/shared_symlink/shared/index.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/before.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/defaultsDeep.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/at.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/defaultsDeepAll.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-7722a2ea/fp/bind.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-b2787570/fp/attempt.js'
  npm WARN tar ENOENT: no such file or directory, open '/home/my_user/BuildAgent/work/my_application/node_modules/.staging/lodash-8fa77886/fp/defaultTo.js'

I have tried cleaning up the node_modules folder, ~/.npm and the package-lock.json before issuing npm install, running the same in different containers, cleaning the checkout folder in teamcity before running npm install, changing the checkout folder and all I can think off. I have also ran npm cache clear --force.

I have also tried running the same steps in a local machine, with the same container, and the compilation is working.

There seems to be some dirty data related to npm or something similar causing this, but I can't find out what.

This is the dockerfile being used:

FROM ubuntu:18.04

USER root

ENV ANDROID_HOME=/android-sdk PATH=$PATH:/android-sdk/tools:/android-sdk/tools/bin:/android-sdk:/platform-tools

RUN apt-get update && apt-get install -y sudo lib32z1 lib32ncurses5 g++ unzip openjdk-8-jdk zsh-common curl gnupg2 git

RUN curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash - && \
apt-get install -y nodejs && \
curl "https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip" -o /tmp/sdk.zip && \
mkdir -p /android-sdk && \
unzip -q /tmp/sdk.zip -d /android-sdk && \
mkdir -p /root/.android/ && touch /root/.android/repositories.cfg && \
rm -rf /tmp/* && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /var/lib/apt/*

RUN echo "export JAVA_OPTS=\"$JAVA_OPTS\"" >> /root/.bashrc && \
echo "export ANDROID_HOME=$ANDROID_HOME" >> /root/.bashrc && \
echo "export PATH=$PATH" >> /root/.bashrc

RUN yes | /android-sdk/tools/bin/sdkmanager --licenses && \
/android-sdk/tools/bin/sdkmanager "tools" "platform-tools" "platforms;android-28" "build-tools;28.0.3" "extras;google;m2repository" "extras;android;m2repository"

RUN yes | npm install [email protected] -g --unsafe-perm && \
tns extension install [email protected] && \
tns usage-reporting disable && \
tns error-reporting enable

RUN nativescript doctor

How can I clean this up?

UPDATE

I have also noticed that npm uninstall loadash triggers the same behaviour and the multiple enoent messages.

Not quite sure why...

Answer 1

Make sure that log output is the end of it.

In my case it was:

npm ERR! code ENOENT
npm ERR! syscall spawn git
npm ERR! path git
npm ERR! errno -2
npm ERR! enoent Error while executing:
npm ERR! enoent undefined ls-remote -h -t ssh://[email protected]/bodymovin/lottie-api.git
npm ERR! enoent 
npm ERR! enoent 
npm ERR! enoent spawn git ENOENT
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

If not, check the end of your log for the actual issue. I experienced this when running a docker container and it turned out one of the packages is being pulled using git as you see in the log, so the git command needs to be in the PATH of the user that runs the npm command.

If it's not that, the end of the log will likely tell you what is the actual error. If it's not clear, it might be the package-lock.json file having old references generated by the package.json when running npm install. Try to remove/rename it and run npm install, then compare the differences.

If that still does not fix it, maybe package.json has a reference that is too old to be resolved by npm.

The npm cache you may be looking for on a Linux system is usually under ~/.npm or ~/.npm-global depending on how you configured npm/node.

If none of this resolves it, please update your answer with a full log, and the exact environment and conditions you run this in. If this only happens for lodash, try to see what happens if you remove it as a dependency. Good luck

Answer 2

When using a private repo as a dependency when your SSH key is not loaded into your SSH agent (likely not stored permanently and you've restarted your system) an npm i can cause these warnings.

eg. this in a package.json

"dependencies": {
    "dependency-name": "git+ssh://[email protected]:projectname/somerepo.git"

Then when installing dependencies I got the npm WARN tar ENOENT: no such file or directory warning for ALL my deps until I re-added my SSH key to my operating system's SSH agent.

Here are my brief notes on SSH keys:

ssh-add -K ~/.ssh/bb_rsa # add an SSH key to your OS' SSH agent
ssh-add ~/.ssh/bb_rsa # add an SSH key to your OS' SSH agent
ssh-add -l bb_rsa # list the keys within your SSH agent

# .ssh/config file
 Host *
   UseKeychain yes
   AddKeysToAgent yes
   IdentityFile ~/.ssh/bb_rsa