Google app engine and paging

Question

How would one go about writing a query that selects items 2000-2010 out of a collection of 10000 objects in the data store.

I know that it can be done like this in GQL:

select * from MyObject limit 10 offset 2000

According to the documentation, when using an offset the engine will still fetch all the rows, only not return them, thus making the query perform in a way that corresponds linearly with the value of offset.

Is there any better way? Such as using a pseudo ROWNUM column like one could do in other types of data stores.

Answer 1

There's no way to efficiently page using offsets, except to cache the results. You can, however, use datastore cursors to implement paging using a 'bookmark' type approach.

Answer 2

Besides using cursors you can also use a sort order approach. For example:

SELECT * FROM MyObject ORDER BY field LIMIT 10;

for the first 10 objects and then for the next 10 objects, etc.

SELECT * FROM MyObject WHERE field > largestFieldValueFromPreviousResult ORDER BY field LIMIT 10;

Field could even be a key if you don't have another appropriate field. Here is a more complete example:

http://code.google.com/appengine/articles/paging.html