How list Amazon S3 bucket contents by modified date?

Question

Most of the time it happens that we load files in a common S3 bucket due to which it becomes hard to figure out data in it.

How can I view objects uploaded on a particular date?

Answer 1

One solution would probably to use the s3api. It works easily if you have less than 1000 objects, otherwise you need to work with pagination.

s3api can list all objects and has a property for the lastmodified attribute of keys imported in s3. It can then be sorted, find files after or before a date, matching a date ...

Examples of running such option

1. all files for a given date

DATE=$(date +%Y-%m-%d) bucket=test-bucket-fh aws s3api list-objects-v2 --bucket "$bucket" \     --query 'Contents[?contains(LastModified, `'"$DATE"'`)]' 

2. all files after a certain date

SINCE=`date --date '-2 weeks +2 days' +%F 2>/dev/null || date -v '-2w' -v '+2d' +%F` #      ^^^^ GNU style                                    ^^^^ BSD style bucket=test-bucket-fh aws s3api list-objects-v2 --bucket "$bucket" \     --query 'Contents[?LastModified > `'"$SINCE"'`]' 

s3api will return a few metadata so you can filter for specific elements

DATE=$(date +%Y-%m-%d) bucket=test-bucket-fh aws s3api list-objects-v2 --bucket "$bucket" \     --query 'Contents[?contains(LastModified, `'"$DATE"'`)].Key' 

Answer 2

Search on a given date

aws s3api list-objects-v2 --bucket BUCKET_NAME --query 'Contents[?contains(LastModified, `YYYY-MM-DD`)].Key' 

Search from a certain date to today

aws s3api list-objects-v2 --bucket BUCKET_NAME  --query 'Contents[?LastModified>=`YYYY-MM-DD`].Key' 

You can optionally remove the .Key from the end of the query to grab all metadata fields from the s3 objects