How to invoke other Cloud Firebase Functions from a Cloud Function

Question

Let's say I have a Cloud Firebase Function - called by a cron job - that produces 30+ tasks every time it's invoked.

These tasks are quite slow (5 - 6 second each in average) and I can't process them directly in the original because it would time out.

So, the solution would be invoking another "worker" function, once per task, to complete the tasks independently and write the results in a database. So far I can think of three strategies:

1. Pubsub messages. That would be amazing, but it seems that you can only listen on pubsub messages from within a Cloud Function, not create one. Resorting to external solutions, like having a GAE instance, is not an option for me.

2. Call the worker http-triggered Firebase Cloud Function from the first one. That won't work, I think, because I would need to wait for a response from the all the invoked worker functions, after they finish and send, and my original Function would time out.

3. Append tasks to a real time database list, then have a worker function triggered by each database change. The worker has to delete the task from the queue afterwards. That would probably work, but it feels there are a lot of moving parts for a simple problem. For example, what if the worker throws? Another cron to "clean" the db would be needed etc.

Another solution that comes to mind is firebase-queue, but its README explicitly states:

"There may continue to be specific use-cases for firebase-queue, however if you're looking for a general purpose, scalable queueing system for Firebase then it is likely that building on top of Google Cloud Functions for Firebase is the ideal route"

It's not officially supported and they're practically saying that we should use Functions instead (which is what I'm trying to do). I'm a bit nervous on using in prod a library that might be abandoned tomorrow (if it's not already) and would like to avoid going down that route.

Answer 1

Sending Pub/Sub messages from Cloud Functions

Cloud Functions are run in a fairly standard Node.js environment. Given the breadth of the Node/NPM ecosystem, the amount of things you can do in Cloud Functions is quite broad.

it seems that you can only listen on pubsub messages from within a Cloud Function, not create one

You can publish new messages to Pub/Sub topics from within Cloud Functions using the regular Node.js module for Pub/Sub. See the Cloud Pub/Sub documentation for an example.

Triggering new actions from Cloud Functions through Database writes

This is also a fairly common pattern. I usually have my subprocesses/workers clean up after themselves at the same moment they write their result back to the database. This works fine in my simple scenarios, but your mileage may of course vary.

If you're having a concrete cleanup problem, post the code that reproduces the problem and we can have a look at ways to make it more robust.