What is the most efficient way to make a batch request to a Firebase DB based on an array of known keys?

Question

I need a solution that makes a Firebase DB API call for multiple items based on keys and returns the data (children) of those keys (in one response).

Since I don't need data to come real-time, some sort of standard REST call made once (rather than a Firebase DB listener), I think it would be ideal.

The app wouldn't have yet another listener and WebSocket connection open. However, I've looked through Firebase's API docs and it doesn't look like there is a way to do this.

Most of the answers I've seen always suggest making a composite key/index of some sort and filter accordingly using the composite key, but that only works for searching through a range. Or they suggest just nesting the data and not worrying about redundancy and disk space (and it's quicker), instead of retrieving associated data through foreign keys.

However, the problem is I am using Geofire and its query method only returns the keys of the items, not the items' data. All the docs and previous answers would suggest retrieving data either by the real-time SDK, which I've tried by using the once method or making a REST call for all items and filter with the orderBy, startAt, endAt params and filtering locally by the keys I need.

This could work, but the potential overhead of retrieving a bunch of items I don't need only to filter them out locally seems wasteful. The approach using the once listener seems wasteful too because it's a server roundtrip for each item key. This approach is kind of explained in this pretty good post, but according to this explanation it's still making a roundtrip for each item (even if it's asynchronously and through the same connection).

This poor soul asked a similar question, but didn't get many helpful replies (that really address the costs of making n number of server requests).

Could someone, once and for all explain the approaches on how this could be done and the pros/cons? Thanks.

Answer 1

Looks like you are looking for Cloud Functions. You can create a function called from http request and do every database read inside of it.

These function are executed in the cloud and their results are sent back to the caller. HTTP call is one way to trigger a Cloud Function but you can setup other methods (schedule, from the app with Firebase SDK, database trigger...). The data are not charged until they leave the server (so only in your request response or if you request a database of another region). Cloud Function billing is based on CPU used, number of invocations and running intances, more details on the quota section.

You will get something like :

const database = require('firebase-admin').database();
const functions = require('firebase-functions');

exports.getAllNodes = functions.https.onRequest((req, res) => {
  
  let children = [ ... ]; // get your node list from req
  let promises = [];

  for (const i in children) {
    promises.push(database.ref(children[i]).once('value'));
  }

  Promise.all(promises)
    .then(result => {
      res.status(200).send(result);
    })
    .catch(error => {
      res.status(503).send(error);
    });
});

That you will have to deploy with the firebase CLI.