Delete after visibility timeout has been reached and the message has been requeued

Question

Let's say I have a job that takes anywhere from 10 seconds to 40 seconds.
The visibility timeout is set at 30 seconds.

A job was finished in 40 seconds, and thus the message went back into the queue, visible to other workers.

The worker who took 40 seconds now deletes the message.

1. Does it success in deleting the message?
2. Will the job be gone from the queue?

Thanks.

Answer 1

If a consumer holds on to a message until after its visibility timeout expires, deleting it does delete it from the queue... usually.

DeleteMessage

Deletes the specified message from the specified queue. You specify the message by using the message's receipt handle and not the MessageId you receive when you send the message. Even if the message is locked by another reader due to the visibility timeout setting, it is still deleted from the queue.

So far, so good, right?

But there's an apparent caveat.

Note

The receipt handle is associated with a specific instance of receiving the message. If you receive a message more than once, the receipt handle you get each time you receive the message is different. If you don't provide the most recently received receipt handle for the message when you use the DeleteMessage action, the request succeeds, but the message might not be deleted.

Exactly why it "might not" be deleted is not explained, but there are a couple of reasons that come to mind:

It may be that SQS doesn't remember an infinite regression of delete handles. If consumers have received the same message, say, 576 different times, it would make sense that (e.g.) the 16th sequentially issued delete handle might long ago have been discarded by SQS since so much time has elapsed...

Or, it could be a quirk of SQS's distributed architecture. For non-FIFO queues, the design of the service is such that in the event of any internal inconsistency, SQS will always prefer to make sure the message is delivered and deleted "at least once," and from an operational perspective, 2 deliveries is far preferrable over 0 deliveries.

The implication here is that the last consumer to receive a message should delete it, and any attempts to delete it by consumers who received it earlier and held on for too long will probably succeed as well.

Quotes are from SQS API Reference: DeleteMessage.