Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Retrieve multiple messages from SQS

I have multiple messages in SQS. The following code always returns only one, even if there are dozens visible (not in flight). setMaxNumberOfMessages I thought would allow multiple to be consumed at once .. have i misunderstood this?

 CreateQueueRequest createQueueRequest = new CreateQueueRequest().withQueueName(queueName);  String queueUrl = sqs.createQueue(createQueueRequest).getQueueUrl();  ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(queueUrl);  receiveMessageRequest.setMaxNumberOfMessages(10);  List<Message> messages = sqs.receiveMessage(receiveMessageRequest).getMessages();  for (Message message : messages) {       // i'm a message from SQS  } 

I've also tried using withMaxNumberOfMessages without any such luck:

 receiveMessageRequest.withMaxNumberOfMessages(10); 

How do I know there are messages in the queue? More than 1?

 Set<String> attrs = new HashSet<String>();  attrs.add("ApproximateNumberOfMessages");  CreateQueueRequest createQueueRequest = new CreateQueueRequest().withQueueName(queueName);  GetQueueAttributesRequest a = new GetQueueAttributesRequest().withQueueUrl(sqs.createQueue(createQueueRequest).getQueueUrl()).withAttributeNames(attrs);  Map<String,String> result = sqs.getQueueAttributes(a).getAttributes();  int num = Integer.parseInt(result.get("ApproximateNumberOfMessages")); 

The above always is run prior and gives me an int that is >1

Thanks for your input

like image 216
sdolgy Avatar asked Mar 31 '12 21:03

sdolgy


People also ask

How do I retrieve messages from SQS?

To receive and delete a message (console)Open the Amazon SQS console at https://console.aws.amazon.com/sqs/ . In the navigation pane, choose Queues. On the Queues page, choose a queue. Choose Send and receive messages.

How many messages can be read from SQS at once?

A single Amazon SQS message queue can contain an unlimited number of messages. However, there is a quota of 120,000 for the number of inflight messages for a standard queue and 20,000 for a FIFO queue.

Can SQS have duplicate messages?

Amazon SQS has a deduplication interval of 5 minutes. Retrying SendMessage requests after the deduplication interval expires can introduce duplicate messages into the queue.

How do I read messages from SQS queue?

When you request messages from a queue, you can't specify which message to retrieve. Instead, you specify the maximum number of messages (up to 10) that you want to retrieve. From the Queues page, select a queue. From Queue Actions, select Send and receive messages.


2 Answers

AWS API Reference Guide: Query/QueryReceiveMessage

Due to the distributed nature of the queue, a weighted random set of machines is sampled on a ReceiveMessage call. That means only the messages on the sampled machines are returned. If the number of messages in the queue is small (less than 1000), it is likely you will get fewer messages than you requested per ReceiveMessage call. If the number of messages in the queue is extremely small, you might not receive any messages in a particular ReceiveMessage response; in which case you should repeat the request.

and

MaxNumberOfMessages: Maximum number of messages to return. SQS never returns more messages than this value but might return fewer.

like image 94
sdolgy Avatar answered Sep 28 '22 04:09

sdolgy


There is a comprehensive explanation for this (arguably rather idiosyncratic) behaviour in the SQS reference documentation.

SQS stores copies of messages on multiple servers and receive message requests are made to these servers with one of two possible strategies,

  • Short Polling : The default behaviour, only a subset of the servers (based on a weighted random distribution) are queried.
  • Long Polling : Enabled by setting the WaitTimeSeconds attribute to a non-zero value, all of the servers are queried.

In practice, for my limited tests, I always seem to get one message with short polling just as you did.

like image 45
Caoilte Avatar answered Sep 28 '22 03:09

Caoilte