AWS Lambda function continues for a bit after context.fail

Question

I have a simple AWS Lambda function that does some validation on images saved to an S3 bucket. I'm using async.waterfall to download the image and process it, but before I even enter the first waterfall function, I do some basic validation on the event data that I am getting from my S3 PutObject trigger, specifically the size information (event.Records[0].s3.object.size). If the event references an image that is greater than my MAX_SIZE, I use context.fail(new Error("Validation error: the file is too big.")) to end execution.

This is all working fine, but I notice in my logs that after the error is logged, the function continues to run for a little bit before it is exited. For example, the first function in my async.waterfall call gets called (i.e. a message from that function shows in the log). I even tried adding a context.done(errorMessage) immediately after the context.fail, and it gets executed (i.e. the message I pass in gets logged) along with a few more lines of code.

Is this expected behavior? I could not find any mention of this in the docs. Does the function take a little bit of time to exit or am I misunderstanding the synchronous nature of the code in the handler function that is BEFORE async.waterfall?

Below is some of my code. All the console.log messages that are shown after context.fail get printed to the log, which I wouldn't expect to happen.

exports.handler = function(event, context) {
  console.log('Received event:', JSON.stringify(event, null, 2));

  if (event.Records[0].s3.object.size > MAX_FILE_SIZE) {
    var error = new Error("Validation error: the file is too big.")
    context.fail(error);
  }
  console.log('Event validation complete.')

  var bucket = event.Records[0].s3.bucket.name;
  var key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
  var fileName = key.split("/")[1];
  var params = {
    Bucket: bucket,
    Key: key
  };
  console.log('Starting processing of the following file: ' + JSON.stringify(params, null, 2));

  async.waterfall([
    function download(callback) {
        // Download the image from S3 into a buffer.
        console.log("Downloading the image from S3...");
        s3.getObject(params, function(err, data) {
            if (err) {
                callback(err);
            } else {
                callback(null, data);
            }
        });
    },
  ...
  ], ...)
}

Answer 1

I believe the aws lambda structure has changed a bit since this question was asked (and James' answer).

Now the handler additionally has a third argument, 'callback', which can be called to stop execution of the script. Depending how you call it, it exits the process successfully or with an error. See those docs for specifics.

Additionally, check out the context property, callbackWaitsForEmptyEventLoop. It defaults to true, but you will want to set it to false so that, once the callback is called, the node process doesn't wait for the runtime loop to clear, i.e. your Async calls are thrown away.

Answer 2

It appears to be expected behavior, even if not well documented. The context documentation does say fail() "indicates failure" but does not promise to stop further execution.

context.fail()

Indicates the Lambda function execution and all callbacks completed unsuccessfully, resulting in a handled exception.

The good news for your code is that it should be fairly easy to return from the function after calling context.fail() to avoid further processing.