If I understand correctly, batch size setting on Lambda decides how many messages to take in one sweep from the SQS. Therefore this JSON (taken from the test Lambda SQS);
{
"Records": [
{
"messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
"receiptHandle": "MessageReceiptHandle",
"body": "FAIL",
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1523232000000",
"SenderId": "123456789012",
"ApproximateFirstReceiveTimestamp": "1523232000001"
},
"messageAttributes": {
},
"md5OfBody": "7b270e59b47ff90a553787216d55d91d",
"eventSource": "aws:sqs",
"eventSourceARN": "arn:aws:sqs:eu-west-1:123456789012:MyQueue",
"awsRegion": "eu-west-1"
}
]
}
There is Records array. And if I set batch size to 5, then if there are 5 messages in SQS, they will be included in array. If there are 10 messages, then Lambda will be invoked twice, with 5 messages in each Record.
I am now confused a bit, as to what approach to take. My Lambda is fairly simple. It will be Axios POST request to external service. If it errors out, I will throw an error. I could even use axios-retry, and make retries fairly easy.
Should I use batch in my case? Naively lookin, all I need is 1 to 1. In other words, message arrives. Lambda takes it. If it errors, it will be retried automatically a bit later.
Contrary, I would have to iterate via all messages and attempt an Axios request. What if the third of five messages fails, in that case I throw an error and Lambda is stopped. What happens to messages four and five? Are they resent to SQS and then again picked up for another execution?
Batch size – The number of records to send to the function in each batch. For a standard queue, this can be up to 10,000 records. For a FIFO queue, the maximum is 10.
By default, Lambda batches up to 10 messages in a queue to process them during a single Lambda execution. You can increase this number up to 10,000 messages, or up to 6 MB of messages in a single batch for standard SQS queues.
Yes there's no reason you can't configure it that way.
all I need is 1 to 1. In other words, message arrives. Lambda takes it. If it errors, it will be retried automatically a bit later.
I don't think you need batch processing in above case.
What if the third of five messages fails, in that case I throw an error and Lambda is stopped. What happens to messages four and five? Are they resent to SQS and then again picked up for another execution?
Message(s) won't be deleted from the queue if your Lambda errors, based on the SQS visibility timeout and redrive policy configuration. SQS will trigger Lambda again according to the configuration. If you have configured DLQ then after maxReceiveCount
is reached, failed messages will be added to the DLQ and removed from the main queue.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With