In node.js, how to use child_process.exec so all can happen asynchronously?

Question

I have a server built on node.js. Below is one of the request handler functions:

var exec = require("child_process").exec

function doIt(response) {

    //some trivial and fast code - can be ignored

    exec(
        "sleep 10",  //run OS' sleep command, sleep for 10 seconds
        //sleeping(10), //commented out. run a local function, defined below.
        function(error, stdout, stderr) {
            response.writeHead(200, {"Content-Type": "text/plain"});
            response.write(stdout);
            response.end();
    });

    //some trivial and fast code - can be ignored
}

Meanwhile, in the same module file there is a local function "sleeping" defined, which as its name indicates will sleep for 10 seconds.

function sleeping(sec) {
    var begin = new Date().getTime();
    while (new Date().getTime() < begin + sec*1000); //just loop till timeup.
}

Here come three questions --

1. As we know, node.js is single-processed, asynchronous, event-driven. Is it true that ALL functions with a callback argument is asynchronous? For example, if I have a function my_func(callback_func), which takes another function as an argument. Are there any restrictions on the callback_func or somewhere to make my_func asynchronous?

2. So at least the child_process.exec is asynchronous with a callback anonymous function as argument. Here I pass "sleep 10" as the first argument, to call the OS's sleep command and wait for 10 seconds. It won't block the whole node process, i.e. any other request sent to another request handler won't be blocked as long as 10 seconds by the "doIt" handler. However, if immediately another request is sent to the server and should be handled by the same "doIt" handler, will it have to wait till the previous "doIt" request ends?

3. If I use the sleeping(10) function call (commented out) to replace the "sleep 10", I found it does block other requests till 10 seconds after. Could anyone explain why the difference?

Thanks a bunch!

-- update per request --

One comment says this question seemed duplicate to another one (How to promisify Node's child_process.exec and child_process.execFile functions with Bluebird?) that was asked one year after this one.. Well these are too different - this was asked for asynchronous in general with a specific buggy case, while that one was asking about the Promise object per se. Both the intent and use cases vary.

(If by any chance these are similar, shouldn't the newer one marked as duplicate to the older one?)

Answer 1

First you can promisify the child_process.

const util = require('util');
const exec = util.promisify(require('child_process').exec);

async function lsExample() {
  const { stdout, stderr } = await exec('ls');
  if (stderr) {
    // handle error
    console.log('stderr:', stderr);
  }
  console.log('stdout:', stdout);

}
lsExample()

As an async function, lsExample returns a promise.

Run all promises in parallel with Promise.all([]).

Promise.all([lsExample(), otherFunctionExample()]);

If you need to wait on the promises to finish in parallel, await them.

await Promise.all([aPromise(), bPromise()]);

If you need the values from those promises

const [a, b] = await Promise.all([aPromise(), bPromise(])

Answer 2

1) No. For example .forEach is synchronous:

var lst = [1,2,3];
console.log("start")
lst.forEach(function(el) {
    console.log(el);
});
console.log("end")

Whether function is asynchronous or not it purely depends on the implementation - there are no restrictions. You can't know it a priori (you have to either test it or know how it is implemented or read and believe in documentation). There's even more, depending on arguments the function can be either asynchronous or synchronous or both.

2) No. Each request will spawn a separate "sleep" process.

3) That's because your sleeping function is a total mess - it is not sleep at all. What it does is it uses an infinite loop and checks for date (thus using 100% of CPU). Since node.js is single-threaded then it just blocks entire server - because it is synchronous. This is wrong, don't do this. Use setTimeout instead.