In Boto3, how to create a Paginator for list_objects with additional keyword arguments?

Question

I'm using a Paginator to iterate over the contents of an S3 bucket (following http://boto3.readthedocs.io/en/latest/guide/paginators.html#creating-paginators):

    client = boto3.client('s3')
    paginator = client.get_paginator('list_objects')
    page_iterator = paginator.paginate(Bucket=<my_bucket>)

    for page in page_iterator:
        for object in page['Contents']:
            key = object['Key']

In this example, the method name 'list_objects' is passed as a string. However, I would actually like to use a 'partial' function list_objects with the keyword argument Prefix="files/", since the objects I'm interested in are in the files/ 'directory' (cf. http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.list_objects).

Is this possible? (It's not clear to me from the documentation how to do this; I could do an if key.startswith("files/") in the double for loop, but this does not seem like an elegant solution).

Answer 1

Did you miss this in the same document? Filtering results

S3.Paginator.list_objects.paginate() accepts a Prefix parameter used to filter the paginated results by prefix server-side before sending them to the client:

client = boto3.client('s3', region_name='us-west-2')
paginator = client.get_paginator('list_objects')
operation_parameters = {'Bucket': 'my-bucket',
                        'Prefix': 'foo/baz'}
page_iterator = paginator.paginate(**operation_parameters)
for page in page_iterator:
    print(page['Contents'])