Efficient way to check whether SQS queue is empty

Question

I have a SQS Queue from which messages are read by multiple hosts. I want to run some job (business logic) after all the messages in the queue have been processed.

How can I check that the queue is empty?

Yes, I can check for ApproximateNumberOfMessages and ApproximateNumberOfMessagesNotVisible queue attributes but these are approximate numbers. I want to stop my hosts polling for messages in the queue when there are no messages left and then run the required job.

Any ideas? Thanks

Answer 1

You could simply note empty receives from the API response while you're polling. Concerning CloudWatch, there is another metric that would be a better fit for this. From the documentation:

NumberOfEmptyReceives

The number of ReceiveMessage API calls that did not return a message.

Units: Count

Valid Statistics: Average, Minimum, Maximum, Sum, Data Samples (displays as Sample Count in the Amazon SQS console)

Some additional info:

• This metric only gets populated every 5 minutes. If you set up an alarm based on this metric, this means your minimum period should be 5 minutes.
• Sum is the most sensible statistic for your use case. If NumberOfEmptyReceives > 0, your polling job checked the queue and received no messages.

I personally used this metric to set up a cloudwatch alarm that will scale down an autoscaling group that hosts my polling job after the sum of NumberOfEmptyReceives > 0 for several consecutive periods. I like doing consecutive periods because it makes it more evident that the queue was not only empty, but has stayed empty.

Answer 2

You could trigger your post business logic on the cloud watch metric related to queue depth. When the depth is 0 then you can send a SNS notification or start a lambda function.

Also this cloud watch metric is better then others since this is actual message count reported by sqs service.