Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a easy way to get Nth page of items from DynamoDB by java?

I am now working on a web app associated with Amazon DynamoDB, I want to achieve a function that my users can directly get to the Nth page to view the item info,

I have been told that the pagination in DynamoDB is based on last key, rather than limit/offset.It doesn't natively support offset.DynamoDB Scan / Query Pagination

Does that mean : If I want to get to the 10th page of items, then I have to query the 9 pages ahead first?(which seems reeeeeally not a good solution)

Is there a easier way to do that?

like image 233
Wenhan Du Avatar asked Feb 08 '17 09:02

Wenhan Du


People also ask

How do you get the item count in DynamoDB?

To get the item count of a dynamodb table, you have to: Open the AWS Dynamodb console and click on your table's name. In the Overview tab scroll down to the Items summary section. Click on the Get live item count button.

How do I get items from DynamoDB table?

To read an item from a DynamoDB table, use the GetItem operation. You must provide the name of the table, along with the primary key of the item you want. The following AWS CLI example shows how to read an item from the ProductCatalog table. With GetItem , you must specify the entire primary key, not just part of it.

Which class would you use to retrieve items from DynamoDB using Java?

To retrieve a single item, you can use the getItem method. Follow these steps: Create an instance of the DynamoDB class. Create an instance of the TableKeysAndAttributes class that describes a list of primary key values to retrieve from a table.

What are the two ways of retrieving data from a DynamoDB table?

In Amazon DynamoDB, you can use either the DynamoDB API, or PartiQL, a SQL-compatible query language, to query an item from a table. With Amazon DynamoDB the Query action lets you retrieve data in a similar fashion. The Query action provides quick, efficient access to the physical locations where the data is stored.


1 Answers

You are right. DynamoDB doesn't support numerical offset. The only way to paginate is to use the LastEvaluatedKey parameter when making a request. You still have some good options to achieve pagination using a number.

Fast Cursor

You can make fast pagination requests by discarding the full result and getting only the Keys. You are limited to 1MB per request. This represents a large amount of Keys! Using this, you can move your cursor to the required position and start reading full objects.

This solution is acceptable for small/medium datasets. You will run into performance and cost issues on large datasets.

Numerical index

You can also create a global secondary index where you will paginate your dataset. You can add for example an offset property to all your objects. You can query this global index directly to get the desired page.

Obviously this only works if you don't use any custom filter... And you have to maintain this value when inserting/deleting/updating objects. So this solution is only good if you have an 'append only' dataset

Cached Cursor

This solution is built on the first one. But instead of fetching keys every single time, you can cache the pages positions and reuse them for other requests. Cache tools like redis or memcached can help you to achieve that.

  1. You check the cache to see if pages are already calculated
  2. If not, you scan your dataset getting only Keys. Then you store the starting Key of each page in your cache.
  3. You request the desired page to fetch full objects

Choose the solution that fits your needs. I hope this will help you :)

like image 105
Sébastien Avatar answered Oct 04 '22 22:10

Sébastien