Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Firestore query reads all the documents again (from database) when only one is modified even with persistence enabled

Expected Behavior: I assumed that when some field of a document is modified, query will only return that document and so it would be considered only 1 read. Say I query a collection with n documents, I would hope that it would be n reads for the initial state, and if a document is modified later only that document would be read again from database, resulting in n+1 reads in total.

What I see is n reads initially, and n reads later whenever one document is modified regardless of Offline Persistence being enalbed or not; resulting in 2n reads with all "fromCache : false" metadata.

  myCollection: 
         doc1 : 
              value: "some string"
         ...
         docn: 
             value: "some text" 

later I change value for one document via console:

     myCollection: 
         doc1 : 
              value: "changed to other value"
         ...
         docn: 
             value: "some text" 

But I get all n documents again in the snapshot of query. Regardless of the fact that I can observe changes using "s.docChanges", it seems that snapshot has all the n documents.

  const someCollectionRef = firebase.firestore().collection("collection/document/subdocl‌​lection"); 
  someCollectionRef.onSnapshot(s => {

      var d = [];
      s.forEach(doc => {

           d.push(doc.data()));

           //fromCache is always false even when one document is 
           //modified it seems that all documents are read from db again 
           console.log(`metadata is ` , doc.metadata);  
      }
      //d seems to have all my documents even when only 1 is modified

  });

No difference with offline persistence enabled. I tested the same example with offline persistence enabled and no difference! Modifying a document still causes all the documents to be read with "fromCache : false" metadata, i.e, they are read from database. The only time data is being read from cache is when a query is refreshed but documents are all identical.

like image 875
TheeBen Avatar asked Jan 28 '18 18:01

TheeBen


1 Answers

I noticed this behavior and did some research: actually all is well!

When you are listening to a collection of N documents, you perform N reads for the initial state, then 1 read when a documents changes, as expected. This is described nicely in this youtube video, and in the documentation for the billing aspect.

The trap is the fromCache metadata. When you receive an update for a document change, all the other unchanged documents have fromCache: false, but they actually come from the cache, as described by a Firebase dev in this SO answer!

The fromCache metadata is true only in the scope of offline persistence, when the initial N reads were done from the local cache. When a document has received data from the server, it sets fromCache: false, and never goes back to fromCache: true. The naming is disturbing, but it seems to be working as expected.

And the code for testing:

<html>
    <body>
        <script src="/__/firebase/7.22.1/firebase-app.js"></script>
        <script src="/__/firebase/7.23.0/firebase-firestore.js"></script>
        <script src="/__/firebase/init.js"></script>
        <script>
            firebase.firestore().enablePersistence().then(() => {
                    const collectionRef = firebase.firestore().collection("/fruits");
                    collectionRef.onSnapshot({includeMetadataChanges: true}, snapshot => {
                    console.log("=================== snapshot received ===================");
                    snapshot.forEach(doc => {
                        console.log(`doc ${doc.id}, metadata:`, doc.metadata);
                    });
                });
            });
        </script>
    </body>
</html>

I modify a document using Firebase console, and in my browser's console, when the long polling request terminates, it shows that a single document was transferred:

15
[[14,["noop"]]]370
[[15,[{
  "documentChange": {
    "document": {
      "name": "projects/xxx/databases/(default)/documents/fruits/apricot",
      "fields": {
        "color": {
          "stringValue": "red"
        }
      },
      "createTime": "2020-10-13T08:14:19.649748Z",
      "updateTime": "2020-10-13T14:16:09.991161Z"
    },
    "targetIds": [
      2
    ]
  }
}
]]]122
[[16,[{
  "targetChange": {
    "resumeToken": "CgkI+bft8+Cx7AI=",
    "readTime": "2020-10-13T14:16:09.991161Z"
  }
}
]]]15
[[17,["noop"]]]
like image 153
Louis Coulet Avatar answered Oct 12 '22 10:10

Louis Coulet