AWS Lambda ending early (without any explicit return or callback)

Question

I'm having a bit of an issue with some node.js code I'm putting into AWS Lambda. I've got a couple of async calls that I need to make, and while the first is behaving like I expect, the lambda function is terminating before the second call is complete.

The return is null, which makes me think that lambda is hitting its implicit callback, but I don't think it should be doing that while there is a promise that hasn't been resolved yet.

Code:

exports.handle = async function(event, context) {
  var AWS = require("aws-sdk");

  AWS.config.update({
    region: "eu-west-1",
    endpoint: "dynamodb.eu-west-1.amazonaws.com"
  });

  var docClient = new AWS.DynamoDB.DocumentClient();
  console.log("Scanning Catalogue...");

  var params = {
    TableName : "Catalogue"
  };

  await docClient.scan(params).promise().then(function (data) {
    console.log("Scan succeeded.");
    data.Items.forEach(function (item) {
      //console.log(item.url);
      checkIfUpdateRequired(item);
    })
  })
}

async function checkIfUpdateRequired (catalogueItem) {
  var request = require("request-promise");
  console.log("Getting " + catalogueItem.url);

  await request(catalogueItem.url).then(function(response) {
    console.log(response.statusCode);
    console.log(response.headers['content-type']);
  });

}

Log output:

START RequestId: 634a55b7-6258-11e8-9f18-6b300c3b5de1 Version: $LATEST
2018-05-28T09:20:44.425Z    634a55b7-6258-11e8-9f18-6b300c3b5de1    Scanning Catalogue...
2018-05-28T09:20:45.446Z    634a55b7-6258-11e8-9f18-6b300c3b5de1    Scan succeeded.
2018-05-28T09:20:47.967Z    634a55b7-6258-11e8-9f18-6b300c3b5de1    Getting https://raw.githubusercontent.com/BSData/wh40k/master/Aeldari%20-%20Craftworlds.cat
2018-05-28T09:20:48.028Z    634a55b7-6258-11e8-9f18-6b300c3b5de1    Getting https://raw.githubusercontent.com/BSData/wh40k/master/Imperium%20-%20Adeptus%20Custodes.cat
END RequestId: 634a55b7-6258-11e8-9f18-6b300c3b5de1
REPORT RequestId: 634a55b7-6258-11e8-9f18-6b300c3b5de1  Duration: 7882.68 ms    Billed Duration: 7900 ms    Memory Size: 128 MB Max Memory Used: 49 MB

So the log tells me that checkIfUpdateRequired() is getting called, but the lambda ends (reporting success with result value null) before the promise is being fulfilled. I'm not doing any manual returns or callbacks to the handler that seem to be the norm for issues related to lambda's ending 'early'.

I'm at my wits end, can anyone offer any suggestions?

Answer 1

You're not waiting for the checkIfUpdateRequired promises to complete; everything in docClient.scan is synchronous in your original code. Use Promise.all to wait for all promises to complete:

await docClient.scan(params).promise().then(function (data) {
  console.log("Scan succeeded.");
  return Promise.all(data.Items.map(checkIfUpdateRequired));
});

Note that if you're using await, your code will be flatter and easier to understand if you habitually use it instead of .then. For example, you could refactor to:

const data = await docClient.scan(params).promise();
return Promise.all(data.Items.map(checkIfUpdateRequired));

and

async function checkIfUpdateRequired (catalogueItem) {
  // actually, better to only require once, rather than on every function call
  const request = require("request-promise");
  console.log("Getting " + catalogueItem.url);
  const response = await request(catalogueItem.url)
  console.log(response.statusCode);
  console.log(response.headers['content-type']);
}