Amazon S3 listing "directories"

Question

I've created a hierarchy in S3 via the AWS S3 Management Console. If I run the following code to list the bucket:

AmazonS3 s3 = new AmazonS3Client(CRED); ListObjectsRequest lor = new ListObjectsRequest()                              .withBucketName("myBucket")                              .withPrefix("code/"); ObjectListing objectListing = s3.listObjects(lor); for (S3ObjectSummary summary: objectListing.getObjectSummaries()) {     System.out.println(summary.getKey()); } 

I get:

code/  code/03000000-0001-0000-0000-000000000000/  code/03000000-0001-0000-0000-000000000000/special.js  code/03000000-0001-0000-0000-000000000000/test.js  code/03000000-0002-0000-0000-000000000000/  

Which is exactly what I would expect. If I add a delimiter though, so that I only list the content directly under "code/" I now don't get any sub "directories" back.

Change line above (add withDelimiter() on the end) to:

ListObjectsRequest lor = new ListObjectsRequest().withBucketName("myBucket")                                                  .withPrefix("code/")                                                  .withDelimiter("/"); 

And I now only get:

code/  

I know that S3 doesn't have "directories", instead delimited keys, but this behaviour seems odd? How would I list what is only immediately below "code"?

Answer 1

Where you have keys that have no content S3 considers them "Common Prefixes":

http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ObjectListing.html#getCommonPrefixes%28%29

public List getCommonPrefixes()

Gets the common prefixes included in this object listing. Common prefixes are only present if a delimiter was specified in the original request.

Each common prefix represents a set of keys in the S3 bucket that have been condensed and omitted from the object summary results. This allows applications to organize and browse their keys hierarchically, similar to how a file system organizes files into directories.

For example, consider a bucket that contains the following keys:

"foo/bar/baz"
"foo/bar/bash"
"foo/bar/bang"
"foo/boo"

If calling listObjects with the prefix="foo/" and the delimiter="/" on this bucket, the returned S3ObjectListing will contain one entry in the common prefixes list ("foo/bar/") and none of the keys beginning with that common prefix will be included in the object summaries list.

Returns: The list of common prefixes included in this object listing, which might be an empty list if no common prefixes were found.