Quick way to list all files in Amazon S3 bucket using php?

Question

I have an amazon s3 bucket that has tens of thousands of filenames in it. What's the easiest way to get a list of all file or text file that lists all the filenames in the bucket?

I have tried with listObject(), but It seems that it only list 1000 files.

amazon-s3-returns-only-1000-entries-for-one-bucket-and-all-for-another-bucket-u S3-Provider-does-not-get-more-than-1000-items-from-bucket

--> Listing Keys Using the AWS SDK for PHP but in aws docs I read

max-keys - string - Optional - The maximum number of results returned by the method call. The returned list will contain no more results than the specified value, but may return fewer. The default value is 1000.

AWS DOC FOR list_objects

Is there some way to list it all and print it to a text file using AWS PHP SDK ?

Possible repeat : quick-way-to-list-all-files-in-amazon-s3-bucket

I have reposted the question because am looking for the solution in php.

Code :

$s3Client = S3Client::factory(array('key' => $access, 'secret' => $secret));

$response = $s3Client->listObjects(array('Bucket' => $bucket, 'MaxKeys' => 1000, 'Prefix' => 'files/'));
$files = $response->getPath('Contents');
$request_id = array();
foreach ($files as $file) {
    $filename = $file['Key'];
    print "\n\nFilename:". $filename;

 }

Answer 1

To get more than 1000 objects, you must make multiple requests using the Marker parameter to tell S3 where you left off for each request. Using the Iterators feature of the AWS SDK for PHP makes it easier to get all of your objects, because it encapsulates the logic of making multiple API requests. Try this:

$objects = $s3Client->getListObjectsIterator(array(
    'Bucket' => $bucket,
    'Prefix' => 'files/'
));

foreach ($objects as $object) {
    echo $object['Key'] . "\n";
}

With latest PHP SDK (as of March 2016) the code must be written like this instead:

$objects = $s3Client->getIterator('ListObjects', array(
    'Bucket' => $bucket,
    'Prefix' => 'files/'
));

Answer 2

Use Paginator to get all files

    $client = new S3Client([
        'version' => AWS_S3_CLIENT_FACTORY_VERSION,
        'region' => AWS_S3_CLIENT_FACTORY_REGION,

    ]);
    $objects = $client->getPaginator('ListObjects', ['Bucket' => "my-bucket"]);
    foreach ($objects as $listResponse) {
        $items = $listResponse->search("Contents[?starts_with(Key,'path/to/folder/')]");
        foreach($items as $item) {
            echo $item['Key'] . PHP_EOL;
        }
    }

To get all files change the search to:

$listResponse->search("Contents[*]");