Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS SQS doesn't reliably trigger Lambda

I've set up a small serverless app using Lambda and SQS. In my case i wanted to trigger a lambda every time a message is added to a SQS Queue.

functions in my serverless.yml

functions:
  collectGame:
    handler: js/collect.collectGame
    memorySize: 128
    timeout: 10
    events:
    - sqs:
        arn:
          Fn::GetAtt:
            - gameRequestQueue
            - Arn
    - http:
        method: post
        cors:
          origin: "https://my-api-url.com"
        path: get/game/{id}
        private: true
        request:
          parameters:
            paths:
              id:true

I tested the process by sending 31 Messages at once to the Queue but realized that only 9 Lambdas get executed (by looking into the cloudwatch logs). I looked into the Queue and can confirm that its being filled with all the messages and that its empty after the 9 Lambdas have been triggered.

I'd expect to have 31 Lambda executions but thats not the case. Anyone knows potential reasons why my Lambdas are not being triggered by the messages?

like image 899
Lucca Avatar asked Aug 21 '18 20:08

Lucca


People also ask

Can Lambda be triggered by SQS?

We can now use Amazon Simple Queue Service (SQS) to trigger AWS Lambda functions!

Is SQS fault tolerance?

SQS lets you decouple application components so that they run and fail independently, increasing the overall fault tolerance of the system. Multiple copies of every message are stored redundantly across multiple Availability Zones so that they are available whenever needed.

Does SQS invoke Lambda asynchronously?

With Amazon SQS, you can offload tasks from one component of your application by sending them to a queue and processing them asynchronously. Lambda polls the queue and invokes your Lambda function synchronously with an event that contains queue messages.

What happens when Lambda fails to process SQS message?

If a Lambda function throws an error, the Lambda service continues to process the failed message until: The message is processed without any error from the function, and the service deletes the message from the queue. The Message retention period is reached and SQS deletes the message from the queue.


Video Answer


1 Answers

Your lambda function is probably being invoked with multiple messages. You should be able to set the BatchSize to 1 when you create the event source mapping, if you only want one message to be sent per lambda invocation

It looks like you are using the serverless framework. See their SQS event documentation for setting the batch size.

like image 157
cementblocks Avatar answered Oct 18 '22 01:10

cementblocks