Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to enumerate all SQS queues in an AWS account

How do I list all SQS queues in an AWS account programmatically via the API and .Net SDK?

I am already doing something similar with DynamoDb tables, and that's fairly straightforward - you can page through results using ListTables in a loop until you have them all.

However the equivalent SQS Api endpoint, ListQueues is different and not as useful. It returns up to 1000 queues, with no option of paging.

Yes, there can be over 1000 queues in my case. I have had a query return exactly 1000 results. It's all in 1 region, so it's not the same as this question.

like image 481
Anthony Avatar asked Oct 31 '16 11:10

Anthony


People also ask

How do I find the SQS queue?

Understanding the Amazon SQS consoleWhen you open the console, choose Queues from the navigation pane to display the Queues page. The Queues page provides information about all of your queues in the active region.

How many SQS queues can I have?

Amazon SQS allows 120,000 inflight messages in standard queues and 20,000 for messages in FIFO queues. There is no limit to the number of message queues a user can create, but the name of the message queue can be no longer than 80 characters.

Can SQS have multiple queues?

Use multiple queues, one per "worker type". Workers should be able to process any message it receives from the queue. Then use something other than SQS to store the result.


1 Answers

You can retrieve SQS queue names from Cloudwatch, which supports paging. It will only return queues that are considered active.

An active queue is described as:

A queue is considered active by CloudWatch for up to six hours from the last activity (for example, any API call) on the queue.

Something like this should work:

var client = new AmazonCloudWatchClient(RegionEndpoint.EUWest1);
string nextToken = null;
var results = Enumerable.Empty<string>();

do
{
    var result = client.ListMetrics(new ListMetricsRequest()
    {
        MetricName = "ApproximateAgeOfOldestMessage",
        NextToken = nextToken
    });

    results = results.Concat(
        result
        .Metrics
        .SelectMany(x => x.Dimensions.Where(d => d.Name == "QueueName")
        .Select(d => d.Value))
    );

    nextToken = result.NextToken;

} while (nextToken != null);
like image 178
Tom Haigh Avatar answered Oct 13 '22 07:10

Tom Haigh