Listing just the sub folders in an s3 bucket

Question

I have an s3 structure as follows:

s3bucketname -> List of first level keys -> List of second level keys -> List of third level keys -> Actual file.

What I need to do is that given the name of the s3bucket and an entry for the first level key, I need the names of all the second level keys that reside under the first level keys. So essentially if we look at it like a folder, I am given the name of the root folder which is the s3bucketname and the name of one of its subfolders subfolder1, I would like to list all the folders that reside within subfolder1. Just the names though, not the complete path.

Can somebody point out how to do it in java using amazon's java sdk?

Thanks

Answer 1

I did the following code which seems to work fine, you have to pass a prefix and make sure the prefix ends with /, and also specify the delimiter you want to get your list of sub-directories. The following should work:

public List<String> listKeysInDirectory(String bucketName, String prefix) {
    String delimiter = "/";
    if (!prefix.endsWith(delimiter)) {
        prefix += delimiter;
    }

    ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
            .withBucketName(bucketName).withPrefix(prefix)
            .withDelimiter(delimiter);
    ObjectListing objects = _client.listObjects(listObjectsRequest);
    return objects.getCommonPrefixes();
}

Answer 2

Charles version is super concise! thanks @charles-menguy

I wrote an extension to support huge list through pagination.

    public List<String> getSubPathsInS3Prefix(String bucketName, String prefix) {
        if (!prefix.endsWith(FILE_DELIMITER)) {
            prefix += FILE_DELIMITER;
        }
        List<String> paths = new ArrayList<String>();
        ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName(bucketName).withPrefix(prefix)
                .withMaxKeys(1000).withDelimiter(FILE_DELIMITER);
        ObjectListing currentListing = s3Client.listObjects(listObjectsRequest);
        paths.addAll(currentListing.getCommonPrefixes());

        while (currentListing == null || currentListing.isTruncated()) {
            currentListing = s3Client.listNextBatchOfObjects(currentListing);
            paths.addAll(currentListing.getCommonPrefixes());
        }
        return paths;
    }

http://www.lazywiz.com/uncategorized/s3-missing-api-list-sub-paths-in-the-s3-bucket/