SQS Messages are moving to DLQ without processing even once

Question

Messages are moving to DLQ without processing even once.

1. Messages available : 1500
2. Messages processed : 1430
3. Moved to DLQ without processing even once : 70

I want to know the reasons why these 70 messages are moved to DLQ even atleast once processing

Checked online/stackoverflow but didn't find anything related to this.

I checked logs for these messages but didn’t find like I found for others. I didn’t open AWS console and saw the messages which will increase max_receives which I didn’t do.

I am using @sqsListener to process 20 messages at a time irrespective of how many messages we get.

So, there no possibility of system hangout/something.

Any help is appreciated.

Answer 1

Too late but I faced same problem. In my case I found few messages in my DLQ where those messageId were not even pulled/processed by my lambda function (why?!?). Aclaration: it was minor percentage (20 over +10k messages) but sometimes it happened.

Searching why ocurred this, I found this paragrahp on AWS documentation:

Because Amazon SQS is a distributed system, it is possible for a consumer to not receive a message even when Amazon SQS marks the message as delivered while returning successfully from a ReceiveMessage API method call.

There is a possibility that few message couldn't been processed by SQS service and then those message are sent to DLQ because are consider as "failed". Thats why AWS suggest us setting the number of maximum receives at least to 5, default value is 10, for a DLQ (in my case was setted to 1).

Upgrating my maxReceiveCount to 5 solved my issue.

Hope this answer helps someone :)