Iterating over the objects in a folder on amazon S3

Question

We have an application where in user can create his own webpages and host them.We are using S3 to store the pages as they are static.Here,as we have a limitation of 100 buckets per user,we decided to go with folders for each user inside a bucket.

Now,if a user wants to host his website on his domain,we ask him for the domain name(when he starts we publish it on our subdomain) and I have to rename the folder.

S3 being a flat file system I know there are actually no folders but just delimeter / separated values so I cannot go into the folder and check how many pages it contains.The API allows it one by one but for that we have to know the object names in the bucket.

I went through the docs and came across iterators,which I have not implemented yet.This uses guzzle of which I have no experience and facing challenges in implementing

Is there any other path I can take or I need to go this way.

Answer 1

You can create an iterator for the contents of a "folder" by doing the following:

$objects = $s3->getIterator('ListObjects', array(
    'Bucket'    => 'bucket-name',
    'Prefix'    => 'subfolder-name/',
    'Delimiter' => '/',
));

foreach ($objects as $object) {
    // Do things with each object
}

If you just need a count, you could this:

echo iterator_count($s3->getIterator('ListObjects', array(
    'Bucket'    => 'bucket-name',
    'Prefix'    => 'subfolder-name/',
    'Delimiter' => '/',
)));

Answer 2

Bit of a learning curve with s3, eh? I spent about 2 hours and ended up with this codeigniter solution. I wrote a controller to loop over my known sub-folders.

function s3GetObjects($bucket) {
    $CI =& get_instance();
    $CI->load->library('aws_s3');

    $prefix = $bucket.'/';

    $objects = $CI->aws_s3->getIterator('ListObjects', array(
        'Bucket'    => $CI->config->item('s3_bucket'),
        'Prefix'    => $prefix,
        'Delimiter' => '/',
    ));

    foreach ($objects as $object) {

        if ($object['Key'] == $prefix) continue;

        echo $object['Key'].PHP_EOL;

        if (!file_exists(FCPATH.$object['Key'])) {

            try {
                $r = $CI->aws_s3->getObject(array(
                    'Bucket' => $CI->config->item('s3_bucket'),
                    'Key'    => $object['Key'],
                    'SaveAs' => FCPATH.$object['Key']
                ));
            } catch (Exception $e) {
                echo $e->getMessage().PHP_EOL;
                //return FALSE;
            }

            echo PHP_EOL;
        } else {
            echo ' -- file exists'.PHP_EOL;
        }
    }

    return TRUE;
}