aws lambda nodejs - error when uploading a zip file compressing by GULP

Question

I'm using Gulp to compress a zip file and then upload it to AWS Lambda. The upload zip file is done manually. Only the process of compressing is handled by Gulp.

Here is my gulpfile.js

var gulp = require('gulp');
var zip = require('gulp-zip');
var del = require('del');
var install = require('gulp-install');
var runSequence = require('run-sequence');
var awsLambda = require("node-aws-lambda");

gulp.task('clean', function() {
  return del(['./dist', './dist.zip']);
});

gulp.task('js', function() {
  return gulp.src('index.js')
    .pipe(gulp.dest('dist/'));
});

gulp.task('npm', function() {
  return gulp.src('./package.json')
    .pipe(gulp.dest('dist/'))
    .pipe(install({production: true}));
});

gulp.task('zip', function() {
  return gulp.src(['dist/**/*', '!dist/package.json'])
    .pipe(zip('dist.zip'))
    .pipe(gulp.dest('./'));
});

gulp.task('deploy', function(callback) {
  return runSequence(
    ['clean'],
    ['js', 'npm'],
    ['zip'],
    callback
  );
});

After running the deploy task, a zip folder named dist.zip is created consists of a index.js file and a node_modules folder. The node_modules folder contains only a lodash library.

This is index.js

var _ = require('lodash');

console.log('Loading function');

exports.handler = (event, context, callback) => {
    //console.log('Received event:', JSON.stringify(event, null, 2));


    var b = _.chunk(['a', 'b', 'c', 'd', 'e'], 3);
    console.log(b);

    callback(null, event.key1);  // Echo back the first key value
    //callback('Something went wrong');
};

After using AWS lambda console to upload the dist.zip folder. There is an error showing that the lodash library cannot be found

{
  "errorMessage": "Cannot find module 'lodash'",
  "errorType": "Error",
  "stackTrace": [
    "Function.Module._load (module.js:276:25)",
    "Module.require (module.js:353:17)",
    "require (internal/module.js:12:17)",
    "Object.<anonymous> (/var/task/index.js:1:71)",
    "Module._compile (module.js:409:26)",
    "Object.Module._extensions..js (module.js:416:10)",
    "Module.load (module.js:343:32)",
    "Function.Module._load (module.js:300:12)",
    "Module.require (module.js:353:17)"
  ]
}

But in the zip folder, there is a node_modules directory that contains the lodash lib.

dist.zip
|---node_modules
    |--- lodash
|---index.js

When i zip the node_modules directory and the file index.js manually, it works fine.

Does anyone have idea what wrongs ? Maybe when compressing using Gulp, there is a misconfigured for the lib path ?

Answer 1

I had same problem few days back.
Everyone pointed to gulp zip, however it was not problem with gulp zip.

Below worked fine:

gulp
    .src(['sourceDir/**'],         {nodir: true, dot: true}  )
    .pipe(zip('target.zip'))
    .pipe(gulp.dest('build/'));

That is, note the below, in 2nd param of src, in the above:

{nodir: true, dot: true}

That is, we have to include dot files for the zip (ex: .config, .abc, etc.)
So, include above in .src of gulp, else all others like copy, zip, etc. will be improper.

Answer 2

The package gulp-zip is massively popular (4.3k downloads per day) and there does not seem to be any Gulp substitute. The problem is definitely with relative paths and how gulp-zip processes them. Even when using a base path option in the gulp.src function (example below), gulp-zip finds a way to mess it up.

gulp.task("default", ["build-pre-zip"], function () {
  return gulp.src([
  "dist/**/*"
  ], { base: "dist/" })
  .pipe(debug())
  .pipe(zip("dist.zip"))
  .pipe(gulp.dest("./dist/"));
});

Since there is no good Gulp solution as of 1/4/2017 I suggest a work-around. I use Gulp to populate the dist folder first, exactly how I need it with the proper node_modules folder. Then it is time to zip the dist folder properly with relative file paths stored. To do that and also update Lambda, I use a batch file (Windows) of command line options to get the job done. Here is the upload.bat file I created to take the place of the gulp-zip task:

start /wait cmd /c "gulp default"
start /wait cmd /c "C:\Program Files\WinRAR\WinRAR.exe" a -r -ep1 dist\dist.zip dist\*.*
aws lambda update-function-code --zip-file fileb://dist/dist.zip --function-name your-fn-name-here

If you use WinRAR you will find their command line docs here, for WinZip go here. That .bat file assumes you are using the AWS Command Line Interface (CLI) which is a godsend; get it here.

If you are wishing this answer pointed you towards a 100% Gulp solution, to that I say, "You and me both!". Good luck.