Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Paginating a DynamoDB query in boto3

How can I loop through all results in a DynamoDB query, if they span more than one page? This answer implies that pagination is built into the query function (at least in v2), but when I try this in v3, my items seem limited:

import boto3
from boto3.dynamodb.conditions import Key, Attr

dynamodb = boto3.resource('dynamodb')
fooTable = dynamodb.Table('Foo')
response = fooTable.query(
    KeyConditionExpression=Key('list_id').eq('123')
)

count = 0

for i in response['Items']:
    count += 1

print count # Prints a subset of my total items
like image 629
Jonathan Avatar asked Sep 06 '16 18:09

Jonathan


People also ask

How does boto3 connect to DynamoDB?

Connecting AWS resources to the python environment requires a boto3 package. Creating a dynamo DB client is a connection instance that lets us connect with our dynamo DB service. We need to specify region_name , aws_access_key_id , aws_secret_access_key in order to connect with our dynamoDb service.

How do I sort DynamoDB Query results?

Query Sort Key DescendingIf the sort key is a number, the results will be returned in numerical order. Else, the results will be returned in UTF-8 bytes order. The sort order is set to ascending by default. However, you can set the ScanIndexForward parameter to false if you need the sort order to be Descending.

Which is faster scan or Query in DynamoDB?

For faster response times, design your tables and indexes so that your applications can use Query instead of Scan . (For tables, you can also consider using the GetItem and BatchGetItem APIs.)


1 Answers

ExclusiveStartKey is the name of the attribute which you are looking for. Use the value that was returned for LastEvaluatedKey in the previous operation.

The data type for ExclusiveStartKey must be String, Number or Binary. No set data types are allowed.

http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html#DynamoDB.Client.query

like image 168
omuthu Avatar answered Nov 01 '22 14:11

omuthu