Reduce the size of the node-modules zip sent by the serverless framework

Question

I am writing a few nodejs functions using the serverless framework. The package.json file requires a few dependencies:

{
  "name": "adam-test-sls",
  "version": "0.1.0",
  "description": "Test package to play with sls/lambda",
  "main": "handler.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Adam Matan <[email protected]>",
  "license": "UNLICENSED",
  "dependencies": {
    "aws-sdk": ">= 2.0.9",
    "json": "^9.0.6",
    "underscore": "^1.8.3",
    "uuid": "^3.1.0"
  },
  "devDependencies": {
    "eslint": "^4.2.0",
    "eslint-config-airbnb": "^15.0.2",
    "eslint-config-airbnb-base": "^11.2.0",
    "eslint-plugin-import": "^2.7.0"
  }
}

The size of the node-modules directory is almost 50mb:

# du -smc node_modules
47  node_modules
47  total

Deployment time is more than 35 seconds, given a zip size of ~9.5MB:

# time serverless deploy function --function hello -v
Serverless: Packaging function: hello...
Serverless: Uploading function: hello (9.46 MB)...
Serverless: Successfully deployed function: hello
serverless deploy function --function hello -v  4.28s user 1.15s system 15% cpu 35.165 total

This is somewhat inefficient - I only change one file, but I have to pack all the unchanged dependencies whenever I make the slightest change.

Any idea how to reduce the zip size (perhaps removing devDependencies), or upload only the changed file?

Answer 1

aws-sdk is in the neighborhood of 24MB and you don't need it since it's already available to lambda functions. One option is to move it to dev-dependencies and then put your dev-dependencies in package.json of a parent directory.

There are also some tools that can help:

serverless-plugin-include-dependencies plugin - thought I'm not sure how well it works if the exclude function is broken: https://github.com/dougmoscrop/serverless-plugin-include-dependencies

Webpack can also be used with serverless-webpack plugin to control dependencies. Webpack's dead code elimination can make a pretty substantial difference.

Not ideal, but you can also run npm prune --production before deployments. (You'll need to run npm install again after.

Answer 2

You can use the serverless ability to exclude some packages you don't need (for example, you're guaranteed to here the manual.

Unfortunately, seems that since version 1.16 there's a problem that those excludes are ignored (version 1.15.1 created much smaller zips, and since 1.16 the zip include anything in your node_modules). I opened an issue, but it is still unanswered.

Answer 3

I used layers for this: Layers are used to pull in additional code and content in the form of layers. Which will keep your lambda functions small and will pull dependencies from layers in runtime.

Step 1: Create a folder somewhere called nodejs

cd Desktop
mkdir nodejs
cd nodejs
npm init

Step 2: In this folder install your dependencies which are large in size. For example:

npm install --save geoip-lite

Step 3: Zip it, give name of your choice.

Step 4: Login to aws console, go to lambda service and then choose create layer. Then upload zip file you created. (As it says you can upload this zip file to S3 in case its larger than 10 mb)

At this step you can assign layer to lambda by going into lambda function and add custom layer which you created from above steps.

In case you are interested to add layer is yml config

functions:
  hello:
   handler: handler.hello
   layers:
    - arn:aws:lambda:region:XXXXXX:layer:LayerName:Y