How to get npm to use cache

Question

UPDATE

I changed directions from this question and ended up taking advantage of Docker image layers to cache the npm install unless there is changes to the package.config, see here.

Note, in relation to this question, I still build my AngularJs Docker image in a slave Jenkins Docker image but I no longer run the npm install in the Docker slave, I copy my app files to my AngularJs Docker image and run the npm install in the AngularJs Docker image, thus getting a Docker cache layer of the npm install, inspiration from this great idea/answer here.

-------------------------------END UPDATE------------------------------

Ok, I should add the caveat that I am in a Docker container but that really shouldn't matter much possibly, I do not stop the container and I have volumes for the for the npm cache folder as well as the /home folder for the user running npm commands.

The purpose of the Docker container, with npm installed, is that it is a build slave, spun up by Jenkins to build an AngularJs application. The problem is that it is incredibly slow, downloading all the needed npm packages, every time.

jenkins is the user, a jenkins account on a build server is "whom" is running npm install

I have Volumes for both the npm folder for the user running the npm install cmd: /home/jenkins/.npm and also the folder that the command npm config get cache says is my cache directory: /root/.npm. Not that container volumes should even matter because I have not stopped the container after running npm install.

Ok the steps I take to start debugging, to start, I "bash into the container" with this command:

docker exec -it <container_id> bash

All commands I run from this point forward I am connected to the running container with npm installed.

echo "$HOME" results in /root

npm config get cache results in root/.npm

Any time jenkins runs npm install in this container, after that command finishes successfully, I run npm cache ls which always yields empty, nothing cached: ~/.npm

Many packages were downloaded however as we can see with ls -a /home/jenkins/.npm/:

So I tried setting the cache-min to a very long expiration time: npm config set cache-min 9999999 that didn't help.

I am not sure what else to do, it just seems that none of my npm packages are being cached, how do I get npm to cache packages?

here is a truncated npm install output:

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.5.3/linux-x64-48_binding.node
Download complete
Binary saved to /home/jenkins/workspace/tsl.frontend.development/node_modules/node-sass/vendor/linux-x64-48/binding.node
Caching binary to /home/jenkins/.npm/node-sass/4.5.3/linux-x64-48_binding.node
Binary found at /home/jenkins/workspace/tsl.frontend.development/node_modules/node-sass/vendor/linux-x64-48/binding.node
Testing binary
Binary is fine
typings WARN deprecated 3/24/2017: "registry:dt/core-js#0.9.7+20161130133742" is deprecated (updated, replaced or removed)
[?25h
+-- app (global)
`-- core-js (global)

And here is my Dockerfile:

FROM centos:7
    MAINTAINER Brian Ogden

    RUN yum update -y && \
             yum clean all

    #############################################
    # Jenkins Slave setup
    #############################################
    RUN yum install -y \
            git \
            openssh-server \
            java-1.8.0-openjdk \
            sudo \
            make && \
            yum clean all

    # gen dummy keys, centos doesn't autogen them like ubuntu does
    RUN /usr/bin/ssh-keygen -A

    # Set SSH Configuration to allow remote logins without /proc write access
    RUN sed -ri 's/^session\s+required\s+pam_loginuid.so$/session optional pam_loginuid.so/' /etc/pam.d/sshd

    # Create Jenkins User
    RUN useradd jenkins -m -s /bin/bash

    # Add public key for Jenkins login
    RUN mkdir /home/jenkins/.ssh
    COPY /files/id_rsa.pub /home/jenkins/.ssh/authorized_keys

    #setup permissions for the new folders and files
    RUN chown -R jenkins /home/jenkins
    RUN chgrp -R jenkins /home/jenkins
    RUN chmod 600 /home/jenkins/.ssh/authorized_keys
    RUN chmod 700 /home/jenkins/.ssh

    # Add the jenkins user to sudoers
    RUN echo "jenkins  ALL=(ALL)  ALL" >> etc/sudoers
    #############################################

    # Expose SSH port and run SSHD
    EXPOSE 22
    #Technically, the Docker Plugin enforces this call when it starts containers by overriding the entry command. 
    #I place this here because I want this build slave to run locally as it would if it was started in the build farm.
    CMD ["/usr/sbin/sshd","-D"]

    #############################################
    # Docker and Docker Compose Install
    #############################################
    #install required packages
    RUN yum install -y \
        yum-utils \
        device-mapper-persistent-data \
        lvm2 \
        curl && \
        yum clean all

    #add Docker CE stable repository
    RUN yum-config-manager \
        --add-repo \
        https://download.docker.com/linux/centos/docker-ce.repo

    #Update the yum package index.
    RUN yum makecache fast

    #install Docker CE
    RUN yum install -y docker-ce-17.06.0.ce-1.el7.centos

    #install Docker Compose 1.14.0
    #download Docker Compose binary from github repo
    RUN curl -L https://github.com/docker/compose/releases/download/1.14.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
    #Apply executable permissions to the binary
    RUN chmod +x /usr/local/bin/docker-compose
    #############################################

    ENV NODE_VERSION 6.11.1

    #############################################
    # NodeJs Install
    #############################################
    RUN yum install -y \
            wget

    #Download NodeJs package
    RUN wget https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.gz

    #extract the binary package into our system's local package hierarchy with the tar command. 
    #The archive is packaged within a versioned directory, which we can get rid of by passing the --strip-components 1 option. 
    #We will specify the target directory of our command with the -C command:
    #This will install all of the components within the /usr/local branch
    RUN tar --strip-components 1 -xzvf node-v* -C /usr/local
    #############################################

#############################################
# npm -setup volume for package cache
# this will speed up builds
#############################################
RUN mkdir /home/jenkins/.npm
RUN chown jenkins /home/jenkins/.npm .
RUN mkdir /root/.npm
RUN chown jenkins /root/.npm .
#for npm cache, this cannot be expressed in docker-compose.yml
#the reason for this is that Jenkins spins up slave containers using
#the docker plugin, this means that there
VOLUME /home/jenkins/.npm
VOLUME /root/.npm
#############################################

Answer 1

When you run docker exec -it <container> bash you access the Docker container as the root user. npm install thus saves the cache to /root/.npm, which isn't a volume saved by the container. Jenkins, on the other hand, uses the jenkins user, which saves to /home/jenkins/.npm, which is being cached. So in order to emulate the functionality of the actual Jenkins workflow, you need to su jenkins before you can npm install.

That being said, the npm cache is not a perfect solution (especially if you have a ton of automated Jenkins builds). Some things to look into that would be better long-term solutions:

• Install a local NPM Cache like sinopia. I found this guide to be particularly helpful.

• Use Docker to build you app (which would work fine with Docker In Docker). Docker would cache after each build step, saving the repeated fetching of dependencies.