Firestore slow queries with large collections unless offline persistence disabled

Question

I am using a Firestore Recycler Adapter to bind to a query of a large collection in firestore.

As I understand the documents in GitHub, the Firestore Recycler Adapter uses .addSnapshotListener(), not the .get() method.

The collection has 5,000 documents, of which I will limit to just 100.

Query query = fsDB.collection("Users").document(user_id).collection("posts")
.orderBy("date_created).orderBy("topic").limit(100);

Now, when I am connected to the internet, and when offline persistence is enabled, this query takes an average of 90 seconds before onDataChanged() is called. I think that is because the recycler is getting data from the memory or cache on the device, then sorting it (!) before getting the data from the network.

BECAUSE, when offline persistence is disabled, the query only takes 2-3 seconds.

How can I use this query to be faster while offline persistence is enabled? Or is there a way to make it so that the listener accesses the network indexes before searching the memory/cache?

Something like...

FirestoreRecyclerOptions<Reed> options = new FirestoreRecyclerOptions.Builder<>()
.setQuery(query, Reed.class)
.setFirst(FirebaseFirestore.getInstance fsDB);

Are the firestore indexes not available on client side?

Using firestore 15.0.0, Android device

Answer 1

Offline queries are not optimized like they are when the app is online. Fast queries are a feature of the Firestore backend which is obviously unavailable when offline.

See also: How reliable is Firestore as an offline persistence mechanism?

Answer 2

@Doug Stevenson's answer (plus comments) gets to the why, but as for a solution, I'd suggest a hybrid approach. Maintain your own local DB or ORM cache, with the desired indexes and possibly your own cache-expunging/cache-limiting scheme.

Use Firestore's built-in persistence for the sync capabilities, and add a Snapshot listener to trigger a refresh of your offline cache. You've got a few optimization options there, from checking only for the delta, to using snapshot.getMetadata().hasPendingWrites() to trigger only when online/fresh data is available.

And of course, you can still implement refresh-on-demand behavior in your UI.

This approach should give you the (self-tuned) offline performance you require, while retaining the benefits of Firestore's back-end.