Including local dependencies in deployment to lambda

Question

I have a repo which consists of several "micro-services" which I upload to AWS's Lambda. In addition I have a few shared libraries that I'd like to package up when sending to AWS.

Therefore my directory structure looks like:

/micro-service-1
    /dist
        package.json
        index.js
/micro-service-2
    /dist
        package.json
        index.js
/shared-component-1
    /dist
        package.json
        component-name-1.js
/shared-component-2
    /dist
        package.json
        component-name-2.js

The basic deployment leverages the handy node-lambda npm module but when I reference a local shared component with a statement like:

var sharedService = require('../../shared-component-1/dist/index');

This works just fine with the node-lambda run command but node-lambda deploy drops this local dependency. Probably makes sense because I'm going below the "root" directory in my dependency so I thought maybe I'd leverage gulp to make this work but I'm pretty darn new to it so I may be doing something dumb. My strategy was to:

• Have gulp deploy depend on a local-deps task
• the local-deps task would:
  • npm build --production to a directory
  • then pipe this directory over to the micro-service under the /local directory
  • clean up the install in the shared

• I would then refer to all shared components like so:

  var sharedService = require('local/component-name-1');
  

Hopefully this makes what I'm trying to achieve. Does this strategy make sense? Is there a simpler way I should be considering? Does anyone have any examples of anything like this in "gulp speak"?

Answer 1

I have an answer to this! :D

TL;DR - Use npm link to link create a symbolic link between your common component and the dependent component.

So, I have a a project with only two modules:

- main-module
- referenced-module

Each of these is a node module. If I cd into referenced-module and run npm link, then cd into main-module and npm link referenced-module, npm will 'install' my referenced-module into my main-module and store it in my node_modules folder. NOTE: When running the second npm link, the name of the project is the one you find in your package.json, not the name of the directory (see npm link documentation, previously linked).

Now, in my main-module all I need to do is var test = require('referenced-module') and I can use that to my hearts content. Be sure to module.exports your code from your referenced-module!

Now, when you zip up main-module to deploy it to AWS Lambda, the links are resolved and the real modules are put in their place! I've tested this and it works, though not with node-lambda yet, though I don't see why this should be a problem (unless it does something different with the package restores).

What's nice about this approach as well is that any changes I make to my referenced-module are automatically picked up by my main-module during development, so I don't have to run any gulp tasks or anything to sync them.

I find this is quite a nice, clean solution and I was able to get it working within a few minutes. If anything I've described above doesn't make any sense (as I've only just discovered this solution myself!), please leave a comment and I'll try and clarify for you.

UPDATE FEB 2016

Depending on your requirements and how large your application is, there may be an interesting alternative that solves this problem even more elegantly than using symlinking. Take a look at Serverless. It's quite a neat way of structuring serverless applications and includes useful features like being able to assign API Gateway endpoints that trigger the Lambda function you are writing. It even allows you to script CloudFormation configurations, so if you have other resources to deploy then you could do so here. Need a 'beta' or 'prod' stage? This can do it for you too. I've been using it for just over a week and while there is a bit of setup to do and things aren't always as clear as you'd like, it is quite flexible and the support community is good!