Does Amazon S3 have a limit to MaxKeys when calling ListObjects?

Question

I always thought there was a 1,000 key limit when calling ListObjects in Amazon S3. However, I just made a call and it's pulling 1,080. But even their docs say there is a limit of 1,000.

I tried setting the MaxKeys setting to 1,000 but it still pulls 1,080 results. My code:

$iterator = $s3->getIterator('ListObjects', array(
    'Bucket' => 'BUCKETNAME',
    'MaxKeys' => 1000
));

It is however pulling folders as keys. But I certainly don't have 80 of them.

Two questions:

1. Is my code wrong?
2. Has Amazon lifted the 1000 key restriction? Is there a new limit?

Thanks in advance!

Answer 1

The S3 API limit hasn't changed, it's still limited to a maximum of 1000 keys/response.

With the PHP SDK v1 a single request returned up to 1000 keys and to get the rest you needed to do a second request with the marker option.

The new PHP SDK (v2) has a concept of Iterators which abstracts the process of doing these multiple, consecutive requests. This makes getting ALL of your objects much easier.

Answer 2

By default the API returns up to 1,000 key names. The response might contain fewer keys but will never contain more. A better implementation would be use the newer ListObjectsV2 API:

 List<S3ObjectSummary> docList=new ArrayList<>();
    ListObjectsV2Request req = new ListObjectsV2Request().withBucketName(bucketName).withPrefix(folderFullPath);
    ListObjectsV2Result listing;
    do{
        listing=this.getAmazonS3Client().listObjectsV2(req);
        docList.addAll(listing.getObjectSummaries());
        String token = listing.getNextContinuationToken();
        req.setContinuationToken(token);
        LOG.info("Next Continuation Token for listing documents is :"+token);
    }while (listing.isTruncated());