How to retrieve paginated children in ascending or descending order in Firebase?

Question

Suppose I'm using firebase for a commenting system and I want to retrieve the comments for a given topic, but there are so many comments in one topic that I don't want to retrieve them all at once. I also want the newest comments to be displayed on top.

It seems that the only way to display firebase records in reverse order is to retrieve them all and then iterate over them in reverse.

This could get very unwieldy on large datasets, especially for mobile clients.

Is there any better way? What is the generic and preferred solution to query paginated data from Firebase, in ascending or descending order?

Answer 1

Updated answer

@Tyler answer stackoverflow.com/a/38024909/681290 is a more up to date and more convenient answer for this question

TL;DR

There is no solution if what you are looking to do is :

ref.orderBy(field, 'DESC').offset(n).limit(x)

Also, in Firebase github there are unsupported tools that do pagination, although only in ascending order only.

Otherwise, here are the closest possible solutions, that I have found myself, or on the web so far. I deliberately interpreted the question as generic, not only about time fields as asked by the OP.

---

Use priorities

The goal is to use setWithPriority() and setPriority() on children to be able to get ordered data later with orderByPriority().

Issues :

• I don't see the advantage over using an indexed priority field instead ? (in fact, priority is stored as an underlying field called .priority, that you can see when exporting json data)
• Hard to maintain for many use cases, to determine value of priority to set

---

Redundant child fields with negative data

For example, we can use a negative timestamp timeRev indexed field in addition to time to be able to get the newest items first.

ref.orderByChild('timeRev')
   .limitToFirst(100);

Issues :

• It adds more complexity to the app : additional fields need to be maintained
• Can break atomicity of the field (for example a score field can be updated at once, not sure if it's possible with two fields, one positive and one negative)
• I think this workaround was used when there was only limit() in the Firebase API, but is kinda obsolete now that we can use limitToFist(), limitToLast(), and range queries (see below)

---

Using limitToLast() and endAt() range queries

This let us avoid negative and redundant field :

ref.orderBy('time')
   .limitToLast(100)

This should be pretty effective with timestamp fields, because usually this is a unique field.

The resulting array of items simply need to be reversed before use. (just remember Array.prototype.reverse() is mutable, so it will change your array)

Issues :

• The API docs say that only the ordered key value can be set as a startAt or endAt boundary. If a lot of items share the same value, the dataset cannot be split in fixed length offsets.

Example with the following items holding a score value :

{
  items: {
    a: {score: 0},
    b: {score: 0},
    c: {score: 1},
    d: {score: 1},
    e: {score: 2},
    f: {score: 5}
  }
}

First page query to get the best scoring items :

ref.child('items').orderByChild('score')
   .limitToLast(3)

Results :

{
  d: {score: 1},
  e: {score: 2},
  f: {score: 5}
}

Note the first item of the subset has a 1 score, so we try to get the previous page by selecting all items with score of 1 or less :

ref.child('items').orderByChild('score')
   .endAt(1)
   .limitToLast(3)

With that query, we get b,c,d items, instead of a,b,c items, which is expected as per the API docs, given the endAt(1) is inclusive, so it will try to get all scores of 1, and has no way to sort which were already returned before.

Workaround

This can be mitigated by not expecting each subset to hold the same amount of record, and discarding those which have already been loaded.

But, if the first million users of my app have a 0 score, this subset cannot be paginated, because the endAt offset is useless, as it is based on the value instead of the number of records.

I don't see any workaround for this use case, I guess Firebase is not intended for this :-)

Edit: In the end, I am using Algolia for all search-related purpose. It's gt a really nice API, and I hope google ends up acquiring Algolia to integrate both Firebase & Algolia, as their complementarity is close to 100%! Disclaimer: no shares owned!! :-)