Cross-compile node module with native bindings with node-gyp

Question

I'm using AWS Lambda, which involves creating an archive of my node.js script, including the node_modules folder and uploading that to their infrastructure to run.

This works fine, except when it comes to node modules with native bindings (using node-gyp). Because the binding was complied and project archived on my local computer (OS X), it is not compatible with AWS's (Amazon Linux) servers.

How can I cross-compile/install a node module (specifically, node-sqlite3) so when I upload it to another server arch it runs?

Answer 1

While not really a solution to your problem, a very easy workaround could be to simply compile the native addons on a Linux machine.

For your particular situation, I would use Vagrant. Vagrant can create virtual machines and configure them within seconds.

1. Find an OS image that resembles Amazon's Linux distro (Fedora, CentOS, others that use yum as package manager - see Wiki)
2. Use a simple configuration script that, when run by Vagrant on machine startup, will run npm install (optionally it might also remove the node_modules folder before to ensure a clean installation)
3. For extra comfort, the script can also create the zip file for deployment
4. Once the installation finishes, the script will shutdown the VM to avoid unnecessary consumption of system resources
5. Deploy!

It might require some tuning if the linked libraries are not at the same place on the target machine but generally this seems to me like the best and quickest solution.

Answer 2

While installing the app using Vagrant might be sufficient in some cases, I have found it necessary to build the app on Linux which is as close to Lambda's Amazon Linux AMI as possible.

You can read the original answer here: https://stackoverflow.com/a/34019739/303184

Steps to make it work:

1. Spawn new EC2 instance. Make sure it is based on exactly the same image as your AWS Lambda runtime. You can review Lambda env details here: http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html. In our case, it was Amazon Linux AMI called amzn-ami-hvm-2015.03.0.x86_64-gp2.

2. Install nvm and use it to install the same version of Node.js as on the AWS Lambda. At the time of writing this, it was v0.10.36. You can refer to http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html again to find out.

3. You will probably need to install git & g++ compiler on the EC2. You can do this running

   sudo yum install git gcc-c++

4. Finally, clone your app to your new EC2 and install your app's dependecies:

   nvm use 0.10.36
   npm install --production
   

5. You can then easily download the node_modules using scp or such.