Menu
I use AWS s3 for storing few thousand files per hour and it works like a charm. I'm curious to see if I can filter them based on time or date or any meta data I have with these objects. I'm able to run a node process to get the objects list and play around for date and time but not meta data. Is there any other way do it or a better option ?.
967
Amazon Athena automatically stores query results and metadata information for each query that runs in a query result location that you can specify in Amazon S3. If necessary, you can access the files in this location to work with them.
Our solution is built with Amazon S3 event notifications, AWS Lambda, AWS Glue Catalog, and Amazon Athena. These services allow you to search thousands of objects in an S3 bucket by filenames, object metadata, and object keys.
Tags are not the same thing as object metadata in S3. Metadata applies only to that object in S3 and cannot be searched on, as you can search with tags.
Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that contains the object. In the Objects list, choose the name of the object for which you want an overview. The object overview opens.
No, you cannot filter on metadata with the S3 API.
To do what you're asking, you would need to List Objects (GET Bucket) on the bucket to get all the keys, then individually ask for metadata for each key (HEAD Object). Then in your own code, you can filter out objects that don't match.
Obviously, this would be very slow to run live if you have more than a few thousand objects. You'll want to either filter down to a manageable number based on prefix or keep an index yourself (elastic search, maybe?). It's common to encode some metadata in the object keys so that you can filter by prefix.
196
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
