Cloud Firestore collection count

Question

As with many questions, the answer is - It depends.

You should be very careful when handling large amounts of data on the front end. On top of making your front end feel sluggish, Firestore also charges you $0.60 per million reads you make.

---

Small collection (less than 100 documents)

Use with care - Frontend user experience may take a hit

Handling this on the front end should be fine as long as you are not doing too much logic with this returned array.

db.collection('...').get().then(snap => {
  size = snap.size // will return the collection size
});

---

Medium collection (100 to 1000 documents)

Use with care - Firestore read invocations may cost a lot

Handling this on the front end is not feasible as it has too much potential to slow down the users system. We should handle this logic server side and only return the size.

The drawback to this method is you are still invoking Firestore reads (equal to the size of your collection), which in the long run may end up costing you more than expected.

Cloud Function:

db.collection('...').get().then(snap => {
  res.status(200).send({length: snap.size});
});

Front End:

yourHttpClient.post(yourCloudFunctionUrl).toPromise().then(snap => {
   size = snap.length // will return the collection size
})

---

Large collection (1000+ documents)

Most scalable solution

---

FieldValue.increment()

As of April 2019 Firestore now allows incrementing counters, completely atomically, and without reading the data prior. This ensures we have correct counter values even when updating from multiple sources simultaneously (previously solved using transactions), while also reducing the number of database reads we perform.

---

By listening to any document deletes or creates we can add to or remove from a count field that is sitting in the database.

See the firestore docs - Distributed Counters Or have a look at Data Aggregation by Jeff Delaney. His guides are truly fantastic for anyone using AngularFire but his lessons should carry over to other frameworks as well.

Cloud Function:

export const documentWriteListener = functions.firestore
  .document('collection/{documentUid}')
  .onWrite((change, context) => {

    if (!change.before.exists) {
      // New document Created : add one to count
      db.doc(docRef).update({ numberOfDocs: FieldValue.increment(1) });
    } else if (change.before.exists && change.after.exists) {
      // Updating existing document : Do nothing
    } else if (!change.after.exists) {
      // Deleting document : subtract one from count
      db.doc(docRef).update({ numberOfDocs: FieldValue.increment(-1) });
    }

    return;
  });

Now on the frontend you can just query this numberOfDocs field to get the size of the collection.

Simplest way to do so is to read the size of a "querySnapshot".

db.collection("cities").get().then(function(querySnapshot) {      
    console.log(querySnapshot.size); 
});

You can also read the length of the docs array inside "querySnapshot".

querySnapshot.docs.length;

Or if a "querySnapshot" is empty by reading the empty value, which will return a boolean value.

querySnapshot.empty;

As far as I know there is no build-in solution for this and it is only possible in the node sdk right now. If you have a

db.collection('someCollection')

you can use

.select([fields])

to define which field you want to select. If you do an empty select() you will just get an array of document references.

example:

db.collection('someCollection').select().get().then( (snapshot) => console.log(snapshot.docs.length) );

This solution is only a optimization for the worst case of downloading all documents and does not scale on large collections!

Also have a look at this:
How to get a count of number of documents in a collection with Cloud Firestore

Be careful counting number of documents for large collections. It is a little bit complex with firestore database if you want to have a precalculated counter for every collection.

Code like this doesn't work in this case:

export const customerCounterListener = 
    functions.firestore.document('customers/{customerId}')
    .onWrite((change, context) => {

    // on create
    if (!change.before.exists && change.after.exists) {
        return firestore
                 .collection('metadatas')
                 .doc('customers')
                 .get()
                 .then(docSnap =>
                     docSnap.ref.set({
                         count: docSnap.data().count + 1
                     }))
    // on delete
    } else if (change.before.exists && !change.after.exists) {
        return firestore
                 .collection('metadatas')
                 .doc('customers')
                 .get()
                 .then(docSnap =>
                     docSnap.ref.set({
                         count: docSnap.data().count - 1
                     }))
    }

    return null;
});

The reason is because every cloud firestore trigger has to be idempotent, as firestore documentation say: https://firebase.google.com/docs/functions/firestore-events#limitations_and_guarantees

Solution

So, in order to prevent multiple executions of your code, you need to manage with events and transactions. This is my particular way to handle large collection counters:

const executeOnce = (change, context, task) => {
    const eventRef = firestore.collection('events').doc(context.eventId);

    return firestore.runTransaction(t =>
        t
         .get(eventRef)
         .then(docSnap => (docSnap.exists ? null : task(t)))
         .then(() => t.set(eventRef, { processed: true }))
    );
};

const documentCounter = collectionName => (change, context) =>
    executeOnce(change, context, t => {
        // on create
        if (!change.before.exists && change.after.exists) {
            return t
                    .get(firestore.collection('metadatas')
                    .doc(collectionName))
                    .then(docSnap =>
                        t.set(docSnap.ref, {
                            count: ((docSnap.data() && docSnap.data().count) || 0) + 1
                        }));
        // on delete
        } else if (change.before.exists && !change.after.exists) {
            return t
                     .get(firestore.collection('metadatas')
                     .doc(collectionName))
                     .then(docSnap =>
                        t.set(docSnap.ref, {
                            count: docSnap.data().count - 1
                        }));
        }

        return null;
    });

Use cases here:

/**
 * Count documents in articles collection.
 */
exports.articlesCounter = functions.firestore
    .document('articles/{id}')
    .onWrite(documentCounter('articles'));

/**
 * Count documents in customers collection.
 */
exports.customersCounter = functions.firestore
    .document('customers/{id}')
    .onWrite(documentCounter('customers'));

As you can see, the key to prevent multiple execution is the property called eventId in the context object. If the function has been handled many times for the same event, the event id will be the same in all cases. Unfortunately, you must have "events" collection in your database.

In 2020 this is still not available in the Firebase SDK however it is available in Firebase Extensions (Beta) however it's pretty complex to setup and use...

A reasonable approach

Helpers... (create/delete seems redundant but is cheaper than onUpdate)

export const onCreateCounter = () => async (
  change,
  context
) => {
  const collectionPath = change.ref.parent.path;
  const statsDoc = db.doc("counters/" + collectionPath);
  const countDoc = {};
  countDoc["count"] = admin.firestore.FieldValue.increment(1);
  await statsDoc.set(countDoc, { merge: true });
};

export const onDeleteCounter = () => async (
  change,
  context
) => {
  const collectionPath = change.ref.parent.path;
  const statsDoc = db.doc("counters/" + collectionPath);
  const countDoc = {};
  countDoc["count"] = admin.firestore.FieldValue.increment(-1);
  await statsDoc.set(countDoc, { merge: true });
};

export interface CounterPath {
  watch: string;
  name: string;
}

Exported Firestore hooks

export const Counters: CounterPath[] = [
  {
    name: "count_buildings",
    watch: "buildings/{id2}"
  },
  {
    name: "count_buildings_subcollections",
    watch: "buildings/{id2}/{id3}/{id4}"
  }
];


Counters.forEach(item => {
  exports[item.name + '_create'] = functions.firestore
    .document(item.watch)
    .onCreate(onCreateCounter());

  exports[item.name + '_delete'] = functions.firestore
    .document(item.watch)
    .onDelete(onDeleteCounter());
});

In action

The building root collection and all sub collections will be tracked.

Here under the /counters/ root path

Now collection counts will update automatically and eventually! If you need a count, just use the collection path and prefix it with counters.

const collectionPath = 'buildings/138faicnjasjoa89/buildingContacts';
const collectionCount = await db
  .doc('counters/' + collectionPath)
  .get()
  .then(snap => snap.get('count'));

Limitations

As this approach uses a single database and document, it is limited to the Firestore constraint of 1 Update per Second for each counter. It will be eventually consistent, but in cases where large amounts of documents are added/removed the counter will lag behind the actual collection count.

I agree with @Matthew, it will cost a lot if you perform such query.

[ADVICE FOR DEVELOPERS BEFORE STARTING THEIR PROJECTS]

Since we have foreseen this situation at the beginning, we can actually make a collection namely counters with a document to store all the counters in a field with type number.

For example:

For each CRUD operation on the collection, update the counter document:

1. When you create a new collection/subcollection: (+1 in the counter) [1 write operation]
2. When you delete a collection/subcollection: (-1 in the counter) [1 write operation]
3. When you update an existing collection/subcollection, do nothing on the counter document: (0)
4. When you read an existing collection/subcollection, do nothing on the counter document: (0)

Next time, when you want to get the number of collection, you just need to query/point to the document field. [1 read operation]

In addition, you can store the collections name in an array, but this will be tricky, the condition of array in firebase is shown as below:

// we send this
['a', 'b', 'c', 'd', 'e']
// Firebase stores this
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e'}

// since the keys are numeric and sequential,
// if we query the data, we get this
['a', 'b', 'c', 'd', 'e']

// however, if we then delete a, b, and d,
// they are no longer mostly sequential, so
// we do not get back an array
{2: 'c', 4: 'e'}

So, if you are not going to delete the collection , you can actually use array to store list of collections name instead of querying all the collection every time.

Hope it helps!

No, there is no built-in support for aggregation queries right now. However there are a few things you could do.

The first is documented here. You can use transactions or cloud functions to maintain aggregate information:

This example shows how to use a function to keep track of the number of ratings in a subcollection, as well as the average rating.

exports.aggregateRatings = firestore
  .document('restaurants/{restId}/ratings/{ratingId}')
  .onWrite(event => {
    // Get value of the newly added rating
    var ratingVal = event.data.get('rating');

    // Get a reference to the restaurant
    var restRef = db.collection('restaurants').document(event.params.restId);

    // Update aggregations in a transaction
    return db.transaction(transaction => {
      return transaction.get(restRef).then(restDoc => {
        // Compute new number of ratings
        var newNumRatings = restDoc.data('numRatings') + 1;

        // Compute new average rating
        var oldRatingTotal = restDoc.data('avgRating') * restDoc.data('numRatings');
        var newAvgRating = (oldRatingTotal + ratingVal) / newNumRatings;

        // Update restaurant info
        return transaction.update(restRef, {
          avgRating: newAvgRating,
          numRatings: newNumRatings
        });
      });
    });
});

The solution that jbb mentioned is also useful if you only want to count documents infrequently. Make sure to use the select() statement to avoid downloading all of each document (that's a lot of bandwidth when you only need a count). select() is only available in the server SDKs for now so that solution won't work in a mobile app.