boto3 find object by metadata or tag

Question

Is it possible to search objects in S3 bucket by object's metadata or tag key/value? (without object name or etag)

I know about head_object() method (ref), but it requires a Key in its parameters.

It seems that get_object() method is also not a solution - it takes the same argument set as head_object(), and nothing about metadata.

As I can see, neither get_* nor list_* methods provide any suitable filters. But I believe that such an opportunity should be in S3 API.

Answer 1

No. The ListObjects() API call does not accept search criteria.

You will need to retrieve a listing of all objects, then call head_object() to obtain metadata.

Alternatively, you could use Amazon S3 Inventory, which can provide a regular CSV file containing a list of all objects and their metadata. Your program could use this as a source of information rather than calling ListObjects().

If you require something that can do real-time searching of metadata, the common practice is to store such information in a database (eg DynamoDB, RDS, Elasticsearch) and then reference the database to identify the desired Amazon S3 objects.