Does Cloud Functions for Firebase make firebase-queue obsolete?

Question

I have written some server-side code to use firebase-queue](https://github.com/firebase/firebase-queue) for scalability, however, with the release of Cloud Functions for Firebase (and its promise of automatic scalability), I am wondering if there is any need for Queue...Has anyone out there combined these two technologies to a greater purpose? Specifically to Firebase developers like @Frank van Puffelen, will Functions replace firebase-queue?

Answer 1

firebaser here

I'm not sure if firebase-queue is not obsolete. Time will have to tell.

But we definitely now use Cloud Functions for Firebase in a lot of scenarios where we'd previously have used firebase-queue and a node worker process. No longer needing to bring our own Node.js process gives an increase in development speed. The auto-scaling of Cloud Functions has proven to be invaluable already.

Combining Cloud Functions with firebase-queue seems illogical. If you simply append nodes to the database and consume them in your function, you have the same behavior without needing the extra library.

Update: one of our database engineers just gave a scenario where Functions can't replace the queue. When making backups, the workers need to mount and unmount remote disks. That task may be possible in a Cloud Function, it is much easier to do in a standalone, self-managed Node process.

Answer 2

I don't think Firebase cloud functions should render firebase-queue obsolete, and I don't think it's illogical at all to want to combine firebase-queue with cloud functions as @@Frank van Puffelen has said.

Firebase-queue provides much more than a way for tasks to listen to firebase and launch tasks. It provides a protocol for communication between parties assigning and and responding to task requests, coordinating retries of failed tasks, reporting progress, state and additional metadata. One of the things this allows is task chaining.

I think it would be useful for Firebase or a third party to develop a firebase-functions-queue package that extends the firebase-functions and allows you to write a firebase cloud function with the same signature as firebase-queue. Something like this:

const functions = require('firebase-functions');
const functions-queue = require('firebase-functions-queue'); //extends firebase-functions with onQueue function

admin.initializeApp(functions.config().firebase);

functions.database.ref('/queue').onQueue(options,function(data,progress,resolve,reject){
    ...
    })

This new package would work just like firebase-queue and use the same metadata and options, except it wouldn't need to deal with managing multiple worker processes since cloud functions already do this automatically and seamlessly.

This would have the following advantages:

• Would allow firebase developers to use a standard way of assigning jobs to queues, monitor progress, failure, etc.
• An app could assign a job to a queue and not care if it was being handled by a cloud function or a different environment.
• A developer could take an existing queue and move it from their node server to cloud functions without changing the client app.
• It could even allow task chaining where one task could be handled by a cloud function and another by a different server within the same job.

Answer 3

I don't think it's as simple as saying it replaces firebase-queue. Ideally, you would use Functions for most of the common use cases for firebase-queue.

However, there are probably also some legitimate reasons to use firebase-queue anyway.

1. It's a pretty robust queuing system that allows you to use multi-state tasks and allows you to set the number of workers.
2. It runs in an environment you specify instead of App Engine (you could install binaries on your server that aren't available in GAE).
3. Could be cheaper depending on specifics of bandwidth vs invocations.
4. It can contact third party endpoints for free, while Functions requires a paid account for these.
5. Functions are still in Beta and don't offer an SLA, they also currently have some startup latencies that wouldn't be experienced in firebase-queue (this point should become invalid around GA release time).