Trying to do fs.writefile() inside an async cloud function (node v8)

Question

I'm currently building a cloud function to scrape some data from ecommerce websites. I'm running them on the Node.js v8 environment, so I'm able to use async/await without transpiling.

I need a way to save a file with the html response from the website so I can inspect it and decide how to best scrape it. And I haven't been able to make it work.

As I found resarching in many places, I need a way to promisify the fs.writeFile() function so that I can await for it. I've tried with util.promisify and also building the custom function below (both produce the same result, which is my cloud function crashing when I await for the result):

index.js (main file for the cloud functions - simplified version)

const functions = require('firebase-functions');
const request = require('request');
const cheerio = require('cheerio');
const fs = require('fs');

// PROMISIFY WRITEFILE TO SAVE URL RESPONSE
const promisifiedWriteFile = function(text) {
  return new Promise((resolve, reject) => {
    fs.writeFile('./response.html', text, err => {
      if (err) {
        return reject(err);
      }
      return resolve();
    });
  });
};

exports.getUrlProductDetails = functions.https.onCall(
  async (data) => {

    try {
      // SEND REQUEST TO URL PRODUCT PAGE, PARSE WITH CHEERIO
      response = await promisifiedRequest(productURL);
      $ = cheerio.load(response.body);

      // TRYING TO SAVE FILE
      await promisifiedWriteFile('12345'); // <----- THIS IS CRASHING
    } // TRY BLOCK - OUTER - END

    catch (err) {
      console.log(err);
      throw new functions.https.HttpsError('unknown', 'Some internal error', err);
    }

  console.log('End of async function...');

  return {
    productDetails: productDetails,
    body: response.body // My client code is getting the body response just fine
  };

Note1: This is a simplified version of the code, but the single line that is crashing the function is: await promisifiedWriteFile('12345'); and when I comment it out, everything else works just fine.

Note2: The file response.html gets created (with empty content), even with the function crashing at that point.

Note3: I should be trying to write the response.body. The '12345' is just for testing. But it crashes either way.

What am I doing wrong?

EDIT

// THIS IS WHAT LOGS OUT OF MY FUNCTION
info: User function triggered, starting execution
info: Execution took 2034 ms, finished with status: 'crash'

And this below is what my client gets: the error message is simply internal, because cloud functions avoid passing internal error details to the client.

Note4: I haven't deployed the function yet. This is all on local development.

Answer 1

I have managed to do it.

According to this Firebase youtube video, the os.tmpdir() only one directory that is writable by your function's code.

So did the following and now it works:

index.js

const fs = require('fs');
const os = require('os');
const path = require('path');
const tmpdir = os.tmpdir();
const filePath = path.join(tmpdir,'response.html');

// PROMISIFY WRITEFILE TO SAVE URL RESPONSE
const promisifiedWriteFile = function(text) {
  return new Promise((resolve, reject) => {
    fs.writeFile(filePath, text, err => {
      if (err) {
        return reject(err);
      }
      return resolve();
    });
  });
};

And from inside the function I'm calling:

await promisifiedWriteFile(response.body);

Found some related doc also on this link:

https://cloud.google.com/functions/docs/concepts/exec