Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Put an "sort_by" on aws-php-sdk ListObjects

In my current project I need to check my S3 bucket contents every 4 seconds for new files.

This script will run for around 3 hours every time that the service is used, and will have something around 2700 files by the end at a single prefix.

This is my function to list those files:

public function listFiles($s3Prefix, $limit, $get_after = ''){
    $command = $this->s3Client->getCommand('ListObjects');
    $command['Bucket']  = $this->s3_bucket;
    $command['Prefix']  = $s3Prefix;
    $command['MaxKeys'] = $limit;
    $command['Marker']  = $s3Prefix.'/'.$get_after;

    //command['Query'] = 'sort_by(Contents,&LastModified)';

    $ret_s3 = $this->s3Client->execute($command);

    $ret['truncated'] = $ret_s3['IsTruncated'];
    $ret['files'] = $ret_s3['Contents'];

    return $ret;
}// listFiles

What I do need is get the files, order by the LastModified field, so I do not need to fetch over 2k files. Is there an extra parameter like

command['Query'] = 'sort_by(Contents,&LastModified)';

to add in the php API?

---------- EDITED ------------

As pointed for Abhishek Meena answer, in the shell it is possible to use

aws s3api list-objects --bucket "bucket-name" --prefix "some-prefix" --query "Contents[?LastModified>=\`2017-03-08\`]"

What I'm looking is how to implement this in PHP.

PHP API: https://github.com/aws/aws-sdk-php

like image 403
Moisés Avatar asked Nov 08 '22 01:11

Moisés


1 Answers

I don't know if they have some thing to sort the objects on the bases of LastModified but you can query and filter objects on the LastModified column. This is what you can use to filter all the files modified after certain time aws s3api list-objects --bucket "bucket-name" --prefix "some-prefix" --query "Contents[?LastModified>=\`2017-03-08\`]"

This is for the shell they might have something similar for the php.

like image 97
Abhishek Meena Avatar answered Jan 04 '23 03:01

Abhishek Meena