Is there a workaround for the Firebase Query "IN" Limit to 10?

Question

I have a query for firebase that has an array of IDs that has a size > 10. Firebase has restrictions on the number of records to query in one session. Is there a way to query against more than 10 at a time?

[Unhandled promise rejection: FirebaseError: Invalid Query. 'in' filters support a maximum of 10 elements in the value array.]

https://cloud.google.com/firestore/docs/query-data/queries

  let query = config.db
    .collection(USER_COLLECTION_NAME)
    .where("id", "in", matchesIdArray);
  const users = await query.get();

(matchesIdArray.length needs to be unlimited)

Answer 1

I found this to work well for me without needing to make as many queries (loop and request in batches of 10).

export async function getContentById(ids, path) {
  // don't run if there aren't any ids or a path for the collection
  if (!ids || !ids.length || !path) return [];

  const collectionPath = db.collection(path);
  const batches = [];

  while (ids.length) {
    // firestore limits batches to 10
    const batch = ids.splice(0, 10);

    // add the batch request to to a queue
    batches.push(
      collectionPath
        .where(
          firebase.firestore.FieldPath.documentId(),
          'in',
          [...batch]
        )
        .get()
        .then(results => results.docs.map(result => ({ /* id: result.id, */ ...result.data() }) ))
    )
  }

  // after all of the data is fetched, return it
  return Promise.all(batches)
    .then(content => content.flat());
}

Answer 2

Your only workaround is to make one query for each item in the array that you would normally use with a single "in" query. Or, batch the requests in the array.

  let query = config.db
    .collection(USER_COLLECTION_NAME)
    .where("id", "==", matchesIdArray[0]);
  const users = await query.get();

You'd have to use the above code in a loop over the matchesIdArray array, and merge the results after they are all done.

Answer 3

BEST METHOD

convert the list into a list that contains sublists of 10 items each. then for loop through that second list and query through firebase in each loop.

EXAMPLE:

List<String> phoneNumbers = ['+12313','+2323','1323','32323','32323','3232', '1232']; //CAN BE UPTO 100 or more

CONVERT THE PHONE NUMBERS TO A LIST OF SUBLIST OF 10 ITEMS EACH

List<List<String>> subList = [];
for (var i = 0; i < phoneNumbers.length; i += 10) {
    subList.add(
        phoneNumbers.sublist(i, i + 10> phoneNumbers.length ? phoneNumbers.length : i + 10));
}

NOW RUN FIREBASE QUERY

subList.forEach((element) {
      firestore
          .collection('Stories')
          .where('userPhone', whereIn: element)
      .get()
      .then((value) {
    
    value.docs.forEach((snapshot) {
      //handle the list
    });

  });