Is there a way to speedup npm ci using cache?

Question

for now npm ci is the most common way to install node modules when using CI. But it is honestly really slow. Is there a way to speedup npm ci using cache or do not fully remove existing packages (whole node_modules folder)?

Answer 1

NPM cache is located in ~/.npm but in most CIs you can only cache things inside your working directory.

What you can do to circumvent this is changing the cache directory to your current directory with npm set cache .npm. The NPM cache will now be located in ./.npm an you can cache this folder between CI jobs.

Example with GitLab CI:

my-super-job:   image: node:13-alpine   script:     - npm set cache .npm     - npm ci    cache:     paths:       - .npm 

EDIT: Just discovered that you can set the config as a command line flag so npm ci --cache .npm should do the same

Answer 2

tl;dr ̶N̶o̶.̶ A little.

npm ci should be preferred in CI because it respects the package-lock.json file. Unlike npm install, which rewrites the file and always installs new versions.

By design this command always purges all local packages, by removing the node_modules directory in the beginning. This is the main reason for long builds. And there is no option to avoid this irritating behaviour.

On a local machine you may speed up npm ci by adding the option --prefer-offline, which tells NPM to ignore the cache minimum time and use locally cached packages right away instead of verifying them against the registry.

However, the NPM cache is located in ~/.npm on Unix, or %AppData%/npm-cache on Windows. These folders are not cacheable by default in most CIs. For example GitLab CI caches only have the repository as available workspace. Other directories like the home directory or system directories (eg. apt) are not cached. Therefore this setting probably wont affect your CI build time.

In older version of NPM the option --progress=false had a significant effect on build time, by removing progress bars. This issue seems to be gone however, I can not measure a noteworthy difference anymore.

A best practice and definitely a gain in speed is to separate packages into production and development. By passing the option --only=production NPM will then ignore development dependencies. Due to the reasons above this wont affect caching.

Update 2021-06: It is now possible to change the cache directory

As pointed out by the comments below, it is now possible to change the location of NPM's cache directory. For example by passing it as argument on every command (--cache .npm) or environment variable (npm_config_cache=.npm). So change this to a path to a temporary directory in your repository, add it to the CI cache stack but exclude it from deployment builds. Then you will be able to use the --prefer-offline argument in CI scripts as well.

Summary:

• Split dependencies
• Set up a local cache directory
• Use npm ci --cache <local cache directory> --prefer-offline --only=production --silent
• This will speed up the process, but due to the command design it can't be as fast as npm install