Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS Lambda finish before sending message to SQS

I'm running a "Node.JS" lambda on AWS that sends a message to SQS. For some reason the SQS callback function get execute only once every couple of calls. It's looks like that the thread that running the lambda finish the run (because it's not a synchronous call to SQS and also can't return a Future) and therefore the lambda doesn't "stay alive" for the callback to get executed.

How can I solve this issue and have the lambda wait for the SQS callback to get execute?

Here is my lambda code:

exports.handler = async (event, context) => {

// Set the region
AWS.config.update({region: 'us-east-1'});

// Create an SQS service object
var sqs = new AWS.SQS({apiVersion: '2012-11-05'});
const SQS_QUEUE_URL = process.env.SQS_QUEUE_URL;

var params = {
    MessageGroupId: "cv",
    MessageDeduplicationId: key,
    MessageBody: "My Message",
    QueueUrl: SQS_QUEUE_URL
};

console.log(`Sending notification via SQS: ${SQS_QUEUE_URL}.`);
sqs.sendMessage(params, function(err, data) {   //<-- This function get called about one time every 4 lambda calls
    if (err) {
        console.log("Error", err);
        context.done('error', "ERROR Put SQS");  
    } else {
        console.log("Success", data.MessageId);
        context.done(null,'');  
    }
});

};
like image 574
Aviram Fireberger Avatar asked May 23 '19 07:05

Aviram Fireberger


People also ask

What happens when Lambda fails to process SQS message?

If a Lambda function throws an error, the Lambda service continues to process the failed message until: The message is processed without any error from the function, and the service deletes the message from the queue. The Message retention period is reached and SQS deletes the message from the queue.

Can AWS Lambda send message to SQS?

You can use a Lambda function to process messages in an Amazon Simple Queue Service (Amazon SQS) queue. Lambda event source mappings support standard queues and first-in, first-out (FIFO) queues.

Does Lambda need to delete SQS message?

Lambda function is used to send received and delete messages from sqs. The code uses the Boto3 to send and receive messages by using AWS.

How does SQS work with Lambda?

You can use an AWS Lambda function to process messages in an Amazon SQS queue. Lambda polls the queue and invokes your Lambda function synchronously with an event that contains queue messages. You can specify another queue to act as a dead-letter queue for messages that your Lambda function can't process.


1 Answers

You should either stick to callback based approach, or to promise based one. I recommend you to use the latter:

exports.handler = async (event, context) => {

  // Set the region
  AWS.config.update({region: 'us-east-1'});

  // Create an SQS service object
  var sqs = new AWS.SQS({apiVersion: '2012-11-05'});
  const SQS_QUEUE_URL = process.env.SQS_QUEUE_URL;

  var params = {
      MessageGroupId: "cv",
      MessageDeduplicationId: key,
      MessageBody: "My Message",
      QueueUrl: SQS_QUEUE_URL
  };

  console.log(`Sending notification via SQS: ${SQS_QUEUE_URL}.`);
  try {
      await sqs.sendMessage(params).promise(); // since your handler returns a promise, lambda will only resolve after sqs responded with either failure or success
  } catch (err) {
    // do something here
  }
};

P.S. Instantiating aws classes in the handler is not a good idea in lambda environment, since it increases the cold start time. It's better to move new AWS.SQS(...) action out of handler and AWS.config.update() too, since these actions will be executed on each call of the handler, but you really need them to be executed only once.

like image 64
Vladyslav Usenko Avatar answered Oct 04 '22 14:10

Vladyslav Usenko