AWS Lambda seems exiting before completion

Question

I have a very simple lambda function (nodeJS) which put the event received in kinesis stream. Here is the source code:

    'use strict';

    const AWS = require('aws-sdk');
    const kinesis = new AWS.Kinesis({apiVersion: '2013-12-02'});

    exports.handler = async (event, context, callback) => {
        let body = JSON.parse(event.body);
        let receptionDate = new Date().toISOString();
        let partitionKey = "pKey-" + Math.floor(Math.random() * 10);

        // Response format needed for API Gateway
        const formatResponse = (status, responseBody) => {
            return {
                statusCode: status,
                headers: { "Content-Type": "application/json" },
                body: JSON.stringify(responseBody)
            }
        }

        // body.events is an array of events. Just add the reception date in each events.
        for(let e of body.events) {
            e.reception_date = receptionDate;
        }

        console.log("put In kinesis stream");
        let kinesisParams = {
            Data: new Buffer(JSON.stringify(body) + "\n"),
            PartitionKey: partitionKey,
            StreamName: 'event_test'
        };

        kinesis.putRecord(kinesisParams, (err, res) => {
            console.log("Kinesis.putRecord DONE");
            if(err) {
                console.log("putRecord Error:", JSON.stringify(err));
                callback(null, formatResponse(500, "Internal Error: " + JSON.stringify(err)));
            } else {
                console.log("putRecord Success:", JSON.stringify(res));
                callback(null, formatResponse(200));
            }
        });
    };

When this code is executed, here are the logs in cloudwatch:

START RequestId: 5d4d7526-1a40-401f-8417-06435f0e5408 Version: $LATEST
2019-01-11T09:39:11.925Z    5d4d7526-1a40-401f-8417-06435f0e5408    put In kinesis stream
END RequestId: 5d4d7526-1a40-401f-8417-06435f0e5408
REPORT RequestId: 5d4d7526-1a40-401f-8417-06435f0e5408  Duration: 519.65 ms Billed Duration: 600 ms     Memory Size: 128 MB Max Memory Used: 28 MB  

It seems that kinesis.putRecord is not called... I don't see anything in kinesis stream logs. I'm certainly wrong somewhere, but I don't know where !

Answer 1

kinesis.putRecord is an asynchronous operation, which calls callback (The second param) when it's finished (whether successful or with an error).

async function is a function that returns a promise. Lambda will finish its execution when this promise is resolved, even if there are other asynchronous operations which are not done yet. Since your function returns nothing, then the promise is immediately resolved when the function ends and therefore the execution will be finished immediately - without waiting to your async kinesis.putRecord task.

When using an async handler, you don't need to call callback. Instead, you return what ever you want, or throw an error. Lambda will get it and respond respectively.

So you have 2 options here:

1. Since you don't have any await in your code, just remove the async. In this case Lambda is waiting for the event loop to be emtpy (Unless you explicitly change context.callbackWaitsForEmptyEventLoop)
2. Change the kinesis.putRecord to something like:

let result;

try {
  result = await kinesis.putRecord(kinesisParams).promise();
} catch (err) {
  console.log("putRecord Error:", JSON.stringify(err));
  throw Error(formatResponse(500, "Internal Error: " + JSON.stringify(err));
}

console.log("putRecord Success:", JSON.stringify(result));
return formatResponse(200);

In the second option, the lambda will keep running until kinesis.putRecord is finished.

For more information about Lambda behavior in this case, you can see the the main code which execute your handler under /var/runtime/node_modules/awslambda/index.js in the lambda container.

Answer 2

@ttulka could you explain a bit more? Give advices or code samples ? – Adagyo

It's about the async processing evolution in JavaScript.

First, everything was done with callback, it's the oldest approach. Using callbacks everywhere leads to "Callback Hell" (http://callbackhell.com).

Then Promises was introduced. Working with Promises looks a bit like working with Monads, everything is packed into a "box" (Promise), so you have to chain all your calls:

thisCallReturnsPromise(...)
  .then(data => ...)
  .then(data => ...)
  .then(data => ...)
  .catch(err => ...)

Which is a bit unnatural to humans, so ECMAScript 2017 proposed a syntactic sugar in async functions (async/await) https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

Async/await syntax allows you to work with async promises like with a normal sync code:

const data = await thisCallReturnsPromise(...)

Don't forget, the await call must be inside an async function:

async () => {
  const data = await thisCallReturnsPromise(...)
  return await processDataAsynchronouslyInPromise(data)
}

AWS Lambda supports Node.js v8.10, which fully implements this syntax.