How can I non-recursively browse the contents of a directory with the AWS S3 API?

Question

Say I have the following directories and files in an Amazon S3 bucket (files are in bold):

• bucketname/
• bucketname/folder1/
• bucketname/folder1/foobar.txt
• bucketname/folder1/subfolder1/
• bucketname/folder1/subfolder1/hello.txt
• bucketname/folder1/subfolder2/
• bucketname/folder1/subfolder2/world.txt
• bucketname/folder1/subfolder2/subsubfolder1/
• bucketname/folder1/subfolder2/subsubfolder1/file.txt

How can I list all objects and immediate subdirectories of a given directory with the .NET AWS S3 API, without recursively getting everything below that directory? In other words, how can I "browse" the contents of a directory at a single level?

For example, imagine I want to browse the contents of bucketname/folder1/. What I would like to see is the following:

• bucketname/folder1/foobar.txt
• bucketname/folder1/subfolder1/
• bucketname/folder1/subfolder2/

...and nothing else. I don't want to list the files and directories in subdirectories, I just want to list the files and subdirectories at the folder1 level.

Is there a way to apply filters to a single AWS API call so that it doesn't return everything and force me to manually parse only what I need?

I've found that this code let's me get just the immediate subdirectories (as intended), but I can't figure out how to include the immediate files too:

var request = new ListObjectsRequest()
    .WithBucketName("bucketname")
    .WithPrefix(@"folder1/")
    .WithDelimiter(@"/");

using (var client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretKey))
using (var response = client.ListObjects(request))
{
    foreach (var item in response.CommonPrefixes)
    {
        /* ... */
    }
}

Answer 1

I had the opposite problem (I knew how to get the files in the specified folder, but not the subdirectories).

The answer is that Amazon lists files differently than it does sub-folders.

Sub-folders are listed, as your example shows, in the ListObjectsResponse.CommonPrefixes collection.

Files are listed in the ListObjectsResponse.S3Objects collection.

So your code should look like this:

var request = new ListObjectsRequest()
.WithBucketName("bucketname")
.WithPrefix(@"folder1/")
.WithDelimiter(@"/");

using (var client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretKey))
using (var response = client.ListObjects(request))
{
    foreach (var subFolder in response.CommonPrefixes)
    {
        /* list the sub-folders */
    }
    foreach (var file in response.S3Objects) {
         /* list the files */
    }
}

my google search turned up this post on the burningmonk blog with this in the comment section:

When you make the Lis­tO­b­jects request, to list the top level fold­ers, don’t set the pre­fix but set the delim­iter to ‘/’, then inspect the ‘Com­mon­Pre­fixes’ prop­erty on the response for the fold­ers that are in the top folder.

To list the con­tents of a ‘root­folder’, make the request with pre­fix set to the name of the folder plus the back­slash, e.g. ‘rootfolder/’ and set the delim­iter to ‘/’. In the response you’ll always have the folder itself as an ele­ment with the same key as the pre­fix you used in the request, plus any sub­fold­ers in the ‘Com­mon­Pre­fixes’ property.