Cache NPM dependencies on Jenkins pipeline

Question

We all know that downloading dependencies with npm can be very time consuming, specially when we are limited to old npm versions.

For me, as a developer, this wasn't such a big deal because I had to do this very few times on my local development machine and everything worked with the node_modules cache in my project's folder. But now I want to take this the applications to a CI environment, with Jenkins.

I realized a huge ammount of time was spent on downloading dependencies with npm. This is a problem because:

1. npm downloads the dependencies in the project's folder, not a global folder such as Maven's /home/user/.m2

2. I have to clean up the Jenkins workspace folder in every run to avoid issues with the git checkout.

I want a very elegant solution for caching the npm dependencies on my Jenkins slaves, but so far I can only think of:

1. Removing everything but the node_modules folders from the Jenkins workspace. I don't like this because I could consume lots of HDD if I keep creating branches for my project. Each branch creates a workspace.

2. doing something like cp ./node_modules /home/npm_cache after every npm install and then cp /home/npm_cache ./node_modules after the code checkout.

I feel these solutions are terrible. There must be a better way to do this.

Answer 1

NPM has a global cache stored in ~/.npm

Answer 2

I created such script to check md5sum of package.json in Jenkins:

stage('NPM Build') {
  steps {
    sh '''
    node -v && npm -v
    '''
    // rm -rf node_modules
    sh '''
    CACHE_FOLDER=${HOME}/.cache/md5
    echo "EXECUTOR_NUMBER: ${EXECUTOR_NUMBER}"
    MD5_FILE_NAME=package-json_${EXECUTOR_NUMBER}.md5sum

    [ -d ${CACHE_FOLDER} ] || mkdir -p ${CACHE_FOLDER}
    ls ${CACHE_FOLDER}

    if [ -f ${CACHE_FOLDER}/${MD5_FILE_NAME} ];then
      cp ${CACHE_FOLDER}/${MD5_FILE_NAME} ${MD5_FILE_NAME}
      md5sum package.json
      cat ${MD5_FILE_NAME}
      md5sum -c ${MD5_FILE_NAME} || npm ci
    else
      echo "No md5sum backup"
      npm ci
    fi

    echo "create new md5sum backup"
    md5sum package.json
    md5sum package.json > ${MD5_FILE_NAME}
    cp ${MD5_FILE_NAME} ${CACHE_FOLDER}
    '''
    sh '''
    npm run ngcc
    '''
    sh '''
    npm run build
    '''
  }
}