Forward and Backward Pagination in DynamoDB

Question

I'm using DynamoDB and NodeJS to enlist some objects on the UI. The list is long, and since DynamoDB can scan/query at most 1MB of data at a time, I've decided to use pagination so at the front end I have Previous and Next buttons to paginate back and forth from the current page.

My problem is I want to retrieve 20 items at a time from Table X by using DynamoDB Query feature based on a chosen DynamoDB Index.
Let's say I've just fetched the initial 20 results (0-20) so on clicking next button I want to fetch results: 21-40 and so on. Also, I want to enable backwards paginating so when I'm on a page which shows results: 41-60, the Back button would again fetch results: 21-40.
As per my understanding, DynamoDB doesn't support numerical offset.

How do I implement backward and forward pagination? I am a newbie in DynamoDB, please help me out.

Answer 1

While DynamoDB will Paginate your data in forward direction only, You'll have to deal with the Backward Pagination in Front End.

For large Tables (exceeding 1MB size), what DynamoDB does:

1. Scans or Queries upto 1MB.
   
2. Returns LastEvaluatedKey to fetch the next set of data or the Next Page. This value is used as Pagination Key in Front End to paginate back and forth.

LastEvaluatedKey holds the value of the last object fetched from DynamoDB during a Scan or Query.

What you need to do (in Back End):

1. Use LIMIT property of DynamoDB Query to specify you want only 20 items.
   
2. Use ExclusiveStartKey property of DynamoDB Query to specify that next set of data would start from the specified value of this property.

What you need to do (in Front End):

1. Keep an array of objects arr[] to host Pagination Keys.

2. Keep a variable page initialized to -1, whose value will indicate the current page user is on.

3. Load the initial page of list into the UI. Now alongside the data, if you have LastEvaluatedKey, push it into the arr and increment page.

Now, you have a single page and page indicates you're on Page 0 and arr contains Page Key of next page.

4. Code of Next Button should follow the logic:

   Request your server to fetch the next page using ExclusiveStartKey = arr[page]

5. When results of next page arrives, you'll again have another LastEvaluatedKey, so again push it into arr and increment page. So you get the picture here, how we save the Page Keys.

6. Code of Back Button should follow the logic:

   Since page variable indicates the Current Page so page - 1 would indicate the previous page. So:
   if (page-1>=0) Request your server to fetch the next page using ExclusiveStartKey = arr[page - 1]

You'll have to manage when Back & Next Buttons are available for clicking by using arr[] and page variables after each page is fetched.

Answer 2

The pagination feature in DynamoDB relies on the LastEvaluatedKey. You should be able to do what you want with that and use the page size to always be 20 items. It just won'y be numerical offset per se.