nodejs formidable async issue

Question

I'm working on a nodejs app using the formidable 3rd party module. And I'm trying to get global scope access (see var formfields, below) to the webform fields returned by the form.parse method below. First, form.parse is an asynch operation and per the module docs here: https://github.com/felixge/node-formidable is designed to take a callback. I'm doing that, see the anonymous function passed to form.parse.

My problem is, no matter what I do I can't get form.parse to properly return the fields object to the formfields variable in global context/scope. I've tried (for days) various combinations of callback functions and closures which return secondary functions, etc. But no luck.

The basic thing I'm trying to do is:

var form = new formidable.IncomingForm();
form.encoding = 'utf-8';
var formfields = form.parse(req, function (err, fields, files) {
  console.log("within form.parse method, subject field of fields object is: " + fields.subjects);
  return fields;
}); // form.parse

console.log("type of formfields is: " + typeof formfields);
console.log("subject field of formfields object is: " + formfields.subjects);

But as you'll see from the console.log output below, even though I'm using a callback, execution is "falling thru" to the last 2 console.log lines of code -- before running the console.log line within form.parse or properly returning "fields" from form.parse and assigning it to 'var formfields'.

Specifically the console.log output shows form.parse returns an object (fields) to become formfields. But it lacks the actual form data (e.g. formfields.subjects) which log as undefined. The last line of output shows form.parse is grabbing the form data (fields.subjects, which logs as "biology") but not properly returning it to become 'var formfields'.

type of formfields is: object
subject field of formfields object is: undefined
within form.parse method, subject field of fields object is: biology

I don't have to have a solution to this as I have the option to just access and manipulate the field data from within form.parse. But I have various things to do with that form data (a db query, results formatting, return to client) and they'd all have to be done nested within form.parse. It's doable but makes for awkward code. I was hoping for a solution to my question as it would make for cleaner code. Thanks in advance for any assistance.

Answer 1

What you are seeing is basic behaviour of asynchronous JavaScript. It looks like you are aware of this but just in case, if it is at all confusing, I suggest you read up on that. A basic Google search for "javascript asynchronicity, callbacks and promises" might be good enough.

Now onto the statement:

It's doable but makes for awkward code.

I guess in some sense it is awkward but this was the way to go some 10 years ago before Promises were introduced by jQuery and later natively by ES2015 and later surpassed by async methods.

Which is probably what you want in your specific case. I will assume that your code is wrapped in some function, like below, otherwise this might not exactly work.

async function yourFunction() {
    var form = new formidable.IncomingForm();
    form.encoding = 'utf-8';
    var formfields = await new Promise(function (resolve, reject) {
        form.parse(req, function (err, fields, files) {
            if (err) {
                reject(err);
                return;
            }
            console.log("within form.parse method, subject field of fields object is: " + fields.subjects);
            resolve(fields);
        }); // form.parse
    });

    console.log("type of formfields is: " + typeof formfields);
    console.log("subject field of formfields object is: " + formfields.subjects);
}

This would force the script to pause until parsing is over. The resolve(fields) function call will ensure that your formFields variable is assigned the value of the fields object. Then the script will resume and your logs will print what you expect them to.

Note that this does not change the way JS works and there still is an asynchronous call happening.

Also note that this will not work natively on NodeJS, so you have to have control over your environment as you would need to do some Babel transpiling. Based on your coding style I would guess that you are not necessarily aware of all of this. If true, you might want to read about new ECMAScripts and how to work with the new features.

Answer 2

For anyone wondering why the correct answer explicitly wraps formidable.incomingForm().parse() in a Promise, formidable does not (yet) support promises natively.

The following will trigger a TypeError: is not a function error because it is looking for the callback function as an argument:

[fields, files]=await form.parse(req)
    .catch(e => console.log(e));

As of Mar 31, 2022 it appears the formidable team is working on a parseAsync option: https://github.com/node-formidable/formidable/issues/685

Until then, an utility function can promisify the call:

const ns= {
    form_parse: async (req, form) => await new Promise((resolve, reject) => form.parse(req, (e, fields, files) => e ? reject(e) : resolve([fields, files])))
}

which allows for async/await flow by passing the form along:

[fields, files]=await ns.form_parse(req, form)
    .catch(e => console.log(e));