Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

delete_message_batch doesn't really delete messages from SQS queue

I am using a standard Amazon SQS queue. Using boto3 library in python3 to interact with SQS. Following is my code for receving messages and then deleting them:

from boto3.session import Session
boto3_session = Session(region_name=SQS_REGION_NAME, aws_access_key_id=SQS_ACCESS_ID,
                                aws_secret_access_key=SQS_ACCESS_KEY)
sqs = boto3_session.client('sqs')

response = sqs.receive_message(
    MessageAttributeNames=[
        'EventToReport',
    ],
    QueueUrl=queue_url,
    MaxNumberOfMessages=10,
    VisibilityTimeout=0,
    WaitTimeSeconds=0
)
messages = response['Messages']
receipt_handles = [{'Id': str(index), 'ReceiptHandle': msg['ReceiptHandle']} for index, msg in enumerate(messages)]
sqs.delete_message_batch(QueueUrl=queue_url, Entries=receipt_handles)

This returns a success response:

{'Successful': [{'Id': '0'}, {'Id': '1'}, {'Id': '2'}, {'Id': '3'}, {'Id': '4'}, {'Id': '5'}, {'Id': '6'}, {'Id': '7'}, {'Id': '8'}, {'Id': '9'}], 'ResponseMetadata': {'RequestId': 'bb28855b-6522-5a1e-a649-d7b3fdabfebe', 'RetryAttempts': 0, 'HTTPStatusCode': 200, 'HTTPHeaders': {'content-length': '1008', 'connection': 'keep-alive', 'server': 'Server', 'date': 'Mon, 29 Jan 2018 03:34:33 GMT', 'content-type': 'text/xml', 'x-amzn-requestid': 'bb28855b-6522-5a1e-a649-d7b3fdabfebe'}}}

But when I look at my SQS management console, I can see that none of the messages are deleted! Visibility timeout for messages was 30 seconds.

I increased the visibility timeout of the queue to 30 minutes, and now only 1 message got deleted although the response was same!

I know it takes time for the delete to complete, so i waited 2-3 minutes every time, but the messages remain in the queue... Am I calling the API in a wrong way? As far as I could find, this is the correct way to call sqs api using boto3.

like image 347
lucifer Avatar asked Jan 29 '18 03:01

lucifer


1 Answers

I finally found the problem with my code. Although I was describing visibility timeout in sqs console, my get request overrode that, so I just removed the following 2 lines from my receive_message code:

VisibilityTimeout=0,
WaitTimeSeconds=0

It is working fine now.

like image 166
lucifer Avatar answered Sep 30 '22 15:09

lucifer