Pagination Techniques using Google App Engine

Question

I want to implement pagination for my website using the Cursor feature of GAE (Java). However, there is only a forward cursor ; backward cursors are not implemented as of App Engine SDK 1.4.0. So, to implement a previous page functionality, it is suggested that I store the cursor page wise in memchache. But my question is - when new record gets added into the datastore, the old cursors for respective pages would become invalid. How do I handle such situations?

Is there anyone who has already implemented this functionality with cursors in Java before? Kindly elaborate the algorithm for this.

Also, I haven't seen a concrete implementation/ example for the same in Java. Could you please share some links if possible.

Answer 1

If your solution is ajax-y, you can keep the cursor (as a string) in an array on the client side in javascript, so you don't need to store it in memcache.

When data gets added (or deleted/changed) the old cursors don't become invalidated. You can still use them. In your case, they basically represent the first item on a page. So the only thing that might happen is if you are on page 3 of results and navigate back and then forward, you might not see the exact same entities you did before on page 3.

For example if you went from page 2 to page 3:

• Page 2 (cursor=x2) results: [d,e,f,...,g]
• Page 3 (cursor=x3) results: [h,i,j,...]

Then, if 'e' got deleted. Going backwards, page 2 (cursor=x2) will now show [d,f,...,g,h], and we update cursor x3 since it changed (we check it after each fetch() for page 2). Going forward, page 3 will now have [i,j,...]

Similarly, if instead 'e2' got added after 'e', going back to page 2 we would have [d,e,e2,f,...] and x3 gets updated. And going forward, page 3 will contain [g,h,i,j,...]

The only caveat is the first page should never use a cursor (in case elements get added before the first one), and you should always allow the user to "try" to go to the next page, in case elements got added after the last result.

So page numbers won't be very specific, but they can't really be when dealing with paged data than can be updated. One trick is to not use page numbers, but label the pages as "data starting with element x" or something like that.

I don't know any implementations, but it should be pretty strait forward to implement. Cursor functionality is described pretty well in the docs: http://code.google.com/appengine/docs/java/datastore/queries.html#Query_Cursors