How to list objects based on prefixes with wildcard using Python Boto3?

Question

I need to find all the files with a specific prefix. For example:

raw/client/Hist/2017/*/*/Tracking_*.zip

I tried this line of code but it does not work:

    import boto3
    client = boto3.client("s3", aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)
    client.list_objects(Bucket="myBucket", Prefix="raw/client/Hist/2017/*/*/Tracking_*.zip")

Answer 1

You won't be able to do this using boto3 without first selecting a superset of objects and then reducing it further to the subset you need via looping. However, you could use Amazon's data wrangler library and the list_objects method, which supports wildcards, to return a list of the S3 keys you need:

import awswrangler as wr
objects = wr.s3.list_objects('s3://myBucket/raw/client/Hist/2017/*/*/Tracking_*.zip')