Deploying node.js project without npm on client side

Question

I would like to deploy a nodejs project with frequent updates. npm is not available at the site so I must package the node_modules. This works ok but takes a long time to send to the customer over the available ftp connection (80MB of mostly node_module files every time). My workflow looks like this:

git clone project
npm install   # installs all my dev tools which I need for packaging
grunt build 
tar xvzf build.tar.gz build/

The build step minfifies my code packaging only what is needed. The node_modules folder is copied into the build folder. If I use npm install --production, I get a smaller footprint but miss the tools I need to build it in the first place. So in the end I go to some effort to make my code footprint small but all my work is undone by having to package such a large node_modules tree.

Is my approach wrong? Is there a simpler way to deploy where npm is not available on the production server or is there a good way to reduce the size of the node_modules folder?

Answer 1

Update: Since writing this answer, npm3 (and yarn) arrived, and flattened npm dependencies. This reduces the size of the node_modules folder considerably (perhaps 20% - 30% for a typical project). Nevertheless, some of the tips below that will reduce your footprint by an order of magnitude.

I have compiled the list of findings for anyone wanting to

1. deploy without npm on the server or
2. reduce the footprint of the node_modules folder

Smaller node_modules footprint:

• Use npm prune --production to remove devDependencies and purge additional modules

  In my case this node_modules folder size by about 20%.

  The bulk of large files under node_modules folders is confined to a small number of modules that are unused at runtime!. Purging/deleting these reduces the footprint by a factor of 10! e.g: karma, bower, less and grunt. Many of these are used by the modules themselves and have no place in a production build. The drawback is that npm install has to be run before each build.

• Use partial npm packages

  Many npm packages are available in parts. For example, of installing all of async or lodash install only the bits you need: e.g.

Bad: npm install -save lodash async

Good: npm install --save async.waterfall async.parallel lodash.foreach

Typically, individual lodash modules are 1/100th the size of the full package.

• npm-package-minifier may be used to reduce the size of the node_modules tree

  Compacting node_modules for client-side deployment

  This basically deletes a lot of unused files in the node_modules tree. This tool will reduce the size of devDependencies also so it should be run on a 'production' version of node_modules.

Reducing size of updates

• Differential deployment

  As mentioned in the comments, updates may be split into updates where dependency changes are required or only business logic changes. I have tried this approach and it greatly reduces the footprint for most updates. However, it also increases the complexity of deployment.