Firestore slow performance issue on getting data

Question

I'm having slow performance issues with Firestore while retrieving basic data stored in a document compared to the realtime database with 1/10 ratio.

Using Firestore, it takes an average of 3000 ms on the first call

 this.db.collection(‘testCol’)    .doc(‘testDoc’)    .valueChanges().forEach((data) => {      console.log(data);//3000 ms later  }); 

Using the realtime database, it takes an average of 300 ms on the first call

 this.db.database.ref(‘/test’).once(‘value’).then(data => {      console.log(data); //300ms later  }); 

This is a screenshot of the network console :

I'm running the Javascript SDK v4.50 with AngularFire2 v5.0 rc.2.

Did anyone experience this issue ?

Answer 1

UPDATE: 12th Feb 2018 - iOS Firestore SDK v0.10.0

Similar to some other commenters, I've also noticed a slower response on the first get request (with subsequent requests taking ~100ms). For me it's not as bad as 30s, but maybe around 2-3s when I have good connectivity, which is enough to provide a bad user experience when my app starts up.

Firebase have advised that they're aware of this "cold start" issue and they're working on a long term fix for it - no ETA unfortunately. I think it's a separate issue that when I have poor connectivity, it can take ages (over 30s) before get requests decide to read from cache.

Whilst Firebase fix all these issues, I've started using the new disableNetwork() and enableNetwork() methods (available in Firestore v0.10.0) to manually control the online/offline state of Firebase. Though I've had to be very careful where I use it in my code, as there's a Firestore bug that can cause a crash under certain scenarios.

---

UPDATE: 15th Nov 2017 - iOS Firestore SDK v0.9.2

It seems the slow performance issue has now been fixed. I've re-run the tests described below and the time it takes for Firestore to return the 100 documents now seems to be consistently around 100ms.

Not sure if this was a fix in the latest SDK v0.9.2 or if it was a backend fix (or both), but I suggest everyone updates their Firebase pods. My app is noticeably more responsive - similar to the way it was on the Realtime DB.

---

I've also discovered Firestore to be much slower than Realtime DB, especially when reading from lots of documents.

Updated tests (with latest iOS Firestore SDK v0.9.0):

I set up a test project in iOS Swift using both RTDB and Firestore and ran 100 sequential read operations on each. For the RTDB, I tested the observeSingleEvent and observe methods on each of the 100 top level nodes. For Firestore, I used the getDocument and addSnapshotListener methods at each of the 100 documents in the TestCol collection. I ran the tests with disk persistence on and off. Please refer to the attached image, which shows the data structure for each database.

I ran the test 10 times for each database on the same device and a stable wifi network. Existing observers and listeners were destroyed before each new run.

Realtime DB observeSingleEvent method:

func rtdbObserveSingle() {      let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))     print("Started reading from RTDB at: \(start)")      for i in 1...100 {         Database.database().reference().child(String(i)).observeSingleEvent(of: .value) { snapshot in             let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))             let data = snapshot.value as? [String: String] ?? [:]             print("Data: \(data). Returned at: \(time)")         }     } } 

Realtime DB observe method:

func rtdbObserve() {      let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))     print("Started reading from RTDB at: \(start)")      for i in 1...100 {         Database.database().reference().child(String(i)).observe(.value) { snapshot in             let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))             let data = snapshot.value as? [String: String] ?? [:]             print("Data: \(data). Returned at: \(time)")         }     } } 

Firestore getDocument method:

func fsGetDocument() {      let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))     print("Started reading from FS at: \(start)")      for i in 1...100 {         Firestore.firestore().collection("TestCol").document(String(i)).getDocument() { document, error in              let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))             guard let document = document, document.exists && error == nil else {                 print("Error: \(error?.localizedDescription ?? "nil"). Returned at: \(time)")                 return             }             let data = document.data() as? [String: String] ?? [:]             print("Data: \(data). Returned at: \(time)")         }     } } 

Firestore addSnapshotListener method:

func fsAddSnapshotListener() {      let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))     print("Started reading from FS at: \(start)")      for i in 1...100 {         Firestore.firestore().collection("TestCol").document(String(i)).addSnapshotListener() { document, error in              let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))             guard let document = document, document.exists && error == nil else {                 print("Error: \(error?.localizedDescription ?? "nil"). Returned at: \(time)")                 return             }             let data = document.data() as? [String: String] ?? [:]             print("Data: \(data). Returned at: \(time)")         }     } } 

Each method essentially prints the unix timestamp in milliseconds when the method starts executing and then prints another unix timestamp when each read operation returns. I took the difference between the initial timestamp and the last timestamp to return.

RESULTS - Disk persistence disabled:

RESULTS - Disk persistence enabled:

Data Structure:

When the Firestore getDocument / addSnapshotListener methods get stuck, it seems to get stuck for durations that are roughly multiples of 30 seconds. Perhaps this could help the Firebase team isolate where in the SDK it's getting stuck?

Answer 2

Update Date March 02, 2018

It looks like this is a known issue and the engineers at Firestore are working on a fix. After a few email exchanges and code sharing with a Firestore engineer on this issue, this was his response as of today.

"You are actually correct. Upon further checking, this slowness on getDocuments() API is a known behavior in Cloud Firestore beta. Our engineers are aware of this performance issue tagged as "cold starts", but don't worry as we are doing our best to improve Firestore query performance.

We are already working on a long-term fix but I can't share any timelines or specifics at the moment. While Firestore is still on beta, expect that there will be more improvements to come."

So hopefully this will get knocked out soon.

---

Using Swift / iOS

After dealing with this for about 3 days it seems the issue is definitely the get() ie .getDocuments and .getDocument. Things I thought were causing the extreme yet intermittent delays but don't appear to be the case:

1. Not so great network connectivity
2. Repeated calls via looping over .getDocument()
3. Chaining get() calls
4. Firestore Cold starting
5. Fetching multiple documents (Fetching 1 small doc caused 20sec delays)
6. Caching (I disabled offline persistence but this did nothing.)

I was able to rule all of these out as I noticed this issue didn't happen with every Firestore database call I was making. Only retrievals using get(). For kicks I replaced .getDocument with .addSnapshotListener to retrieve my data and voila. Instant retrieval each time including the first call. No cold starts. So far no issues with the .addSnapshotListener, only getDocument(s).

For now, I'm simply dropping the .getDocument() where time is of the essence and replacing it with .addSnapshotListener then using

for document in querySnapshot!.documents{ // do some magical unicorn stuff here with my document.data() } 

... in order to keep moving until this gets worked out by Firestore.