How do you structure sequential AWS service calls within lambda given all the calls are asynchronous?

Question

I'm coming from a java background so a bit of a newbie on Javascript conventions needed for Lambda.

I've got a lambda function which is meant to do several AWS tasks in a particular order, depending on the result of the previous task.

Given that each task reports its results asynchronously, I'm wondering if the right way make sure they all happen in the right sequence, and the results of one operation are available to the invocation of the next function.

It seems like I have to invoike each function in the callback of the prior function, but seems like that will some kind of deep nesting and wondering if that is the proper way to do this.

For example on of these functions requires a DynamoDB getItem, following by a call to SNS to get an endpoint, followed by a SNS call to send a message, followed by a DynamoDB write.

What's the right way to do that in lambda javascript, accounting for all that asynchronicity?

Answer 1

I like the answer from @jonathanbaraldi but I think it would be better if you manage control flow with Promises. The Q library has some convenience functions like nbind which help convert node style callback API's like the aws-sdk into promises.

So in this example I'll send an email, and then as soon as the email response comes back I'll send a second email. This is essentially what was asked, calling multiple services in sequence. I'm using the then method of promises to manage that in a vertically readable way. Also using catch to handle errors. I think it reads much better just simply nesting callback functions.

var Q = require('q');
var AWS = require('aws-sdk');    
AWS.config.credentials = { "accessKeyId": "AAAA","secretAccessKey": "BBBB"};
AWS.config.region = 'us-east-1';

// Use a promised version of sendEmail
var ses = new AWS.SES({apiVersion: '2010-12-01'});
var sendEmail = Q.nbind(ses.sendEmail, ses);

exports.handler = function(event, context) {

    console.log(event.nome);
    console.log(event.email);
    console.log(event.mensagem);

    var nome = event.nome;
    var email = event.email;
    var mensagem = event.mensagem;

    var to = ['[email protected]'];
    var from = '[email protected]';

    // Send email
    mensagem = ""+nome+"||"+email+"||"+mensagem+"";

    console.log(mensagem);

    var params = { 
        Source: from, 
        Destination: { ToAddresses: to },
        Message: {
        Subject: {
            Data: 'Form contact our Site'
        },
        Body: {
            Text: {
                Data: mensagem,
            }
        }
    };

    // Here is the white-meat of the program right here.
    sendEmail(params)
        .then(sendAnotherEmail)
        .then(success)
        .catch(logErrors);

    function sendAnotherEmail(data) {
        console.log("FIRST EMAIL SENT="+data);

        // send a second one.
        return sendEmail(params);
    }

    function logErrors(err) {
        console.log("ERROR="+err, err.stack);
        context.done();
    }

    function success(data) {
        console.log("SECOND EMAIL SENT="+data);
        context.done();
    }
}

Answer 2

Short answer:

Use Async / Await — and Call the AWS service (SNS for example) with a .promise() extension to tell aws-sdk to use the promise-ified version of that service function instead of the call back based version.

Since you want to execute them in a specific order you can use Async / Await assuming that the parent function you are calling them from is itself async.

For example:

let snsResult = await sns.publish({
    Message: snsPayload,
    MessageStructure: 'json',
    TargetArn: endPointArn
}, async function (err, data) {
    if (err) {
        console.log("SNS Push Failed:");
        console.log(err.stack);
        return;
    }
    console.log('SNS push suceeded: ' + data);
    return data;
}).promise();

The important part is the .promise() on the end there. Full docs on using aws-sdk in an async / promise based manner can be found here: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/using-promises.html

In order to run another aws-sdk task you would similarly add await and the .promise() extension to that function (assuming that is available).

For anyone who runs into this thread and is actually looking to simply push promises to an array and wait for that WHOLE array to finish (without regard to which promise executes first) I ended up with something like this:

let snsPromises = [] // declare array to hold promises
let snsResult = await sns.publish({
    Message: snsPayload,
    MessageStructure: 'json',
    TargetArn: endPointArn
}, async function (err, data) {
    if (err) {
        console.log("Search Push Failed:");
        console.log(err.stack);
        return;
    }
    console.log('Search push suceeded: ' + data);
    return data;
}).promise();

snsPromises.push(snsResult)
await Promise.all(snsPromises)

Hope that helps someone that randomly stumbles on this via google like I did!