NPM caching similar to a local Maven cache

Question

Gradle's dependency management system stores downloaded artifacts in a local Maven cache. When a build requests that same dependency again the dependency is simply retrieved from the cache, avoiding any network transfer of the artifact.

I'm trying to replicate this behavior with NPM for building JavaScript projects. I was expecting NPM to support a global node_modules cache, but installing a package "globally" in NPM has a different meaning => the package is added to PATH so that it can be used as a CLI tool.

Reading the documenation for npm install, the standard behavior is to install packages into a local node_modules directory. But this would mean many duplicated packages on the system wasting valuable disk space. It also poses a problem for doing clean production builds, since ideally the node_modules should be blown away each time.

Does NPM support something like the Gradle's Maven caching? Documentation on NPM cache doesn't make it any clearer how this is to be used. What's more, it's not obvious if a caching strategy with NPM is safe across multiple parallel builds.

This seems like such a basic requirement for busy CI environments that it must have been solved before. I found the npm-cache tool which seems to offer this support, but it would be much better if caching was supported natively in npm itself.

Thanks!

Answer 1

IMHO it is a pity that the makers did not learn from things like maven that have already been there. If you are doing microservices and have many apps on your machine and you might also have multiple branches or a local jenkins you will have each dependency N*M times on the disk what is an extraordinary waste of disc-space and performance. So you have to be aware that Java or .NET/C# are mature ecosystems while the JavaScript ecosystem is still in the childhood with lots of flaws and edges. But JavaScript is evolving fast so lets hope for the best. Feel free to discuss your pain with the npm makers (https://github.com/npm/npm/issues/).

However, a partial cure comes if you go away from npm and switch to yarn: http://yarnpkg.com/

Answer 2

NPM Cache already comes bundled with NPM out of the box(listed under cli commands). And its main utility is to avoid the network transfer of the same package over and over.

Regarding the duplicate packages issue, as of npm v3 there has been an effort in terms of finding ways to deduplicate dependencies. But it still does not work exactly like Gradle since it is still possible to end up with duplicates of the same package in your node_modules folder.

Per NPM documentation:

Your node_modules directory structure and therefore your dependency tree are dependant on install order

Although a fresh npm install from the same package json always produces the same dependency tree:

The npm install command, when used exclusively to install packages from apackage.json, will always produce the same tree. This is because install order from a package.json is always alphabetical. Same install order means that you will get the same tree.

So at least there is a way to get consistent dependency trees, albeit there's no guarantee it will be the most efficient one. At least those differences do not interfere correct functioning of NPM.

Hope that helps.