spawnSync /bin/sh ENOBUFS

Question

Error "spawnSync /bin/sh ENOBUFS" spawns in my NodeJs application non-systematically while executing the following line :

child_process.execSync(`cd /tmp/myFolder ; tar -xjf myArchive.tar.bz2`);

Archive dimension is 81.5 MB, NodeJs version with NVM : 12.17.0.

Answer 1

The problem is that execSync mode execute the command within a shell with a limited buffer (200 Kb) used to forward the execution output. Moreover, the default shell execution option is "pipe", which means that the output must be forwarded to the parent.

In order to let the shell ignore the execution output, i.e. forward to /dev/null the output, and hence prevent the buffer from filling up, you must use the "ignore" execution option as following :

child_process.execSync(`cd /tmp/myFolder ; tar -xjf myArchive.tar.bz2`, { stdio: 'ignore' });

Read more about exec and spawn execution modes here and here

P.S. Also consider that this error spawns systematically when, during an archive extraction, you run out of disk space.

Answer 2

Use child_process.spawn if the output is important to you.

And even if you don't need it, execution output can be helpful when debugging. You can always add a simple switch which lets you silence the output should you choose.

I try not to use child_process.exec because child_process.spawn works just as fine but without the limitations of exec. Of course, YMMV if your core application logic deals with streaming data. The reason child_process.spawn will work here is because it streams output whereas child_process.exec buffers output and if you fill up the max buffer then child_process.exec will crash. You could increase the buffer size of child_process.exec via the options parameter but remember, the bigger the buffer, the more memory you use, whereas streaming usually keeps the memory usage to a minimum.

Here's a reference implementation. FYI, This code works on Node v14.

const child_process = require("child_process");

function spawn(instruction, spawnOpts = {}, silenceOutput = false) {
    return new Promise((resolve, reject) => {
        let errorData = "";

        const [command, ...args] = instruction.split(/\s+/);

        if (process.env.DEBUG_COMMANDS === "true") {
            console.log(`Executing \`${instruction}\``);
            console.log("Command", command, "Args", args);
        }

        const spawnedProcess = child_process.spawn(command, args, spawnOpts);

        let data = "";

        spawnedProcess.on("message", console.log);

        spawnedProcess.stdout.on("data", chunk => {
            if (!silenceOutput) {
                console.log(chunk.toString());
            }

            data += chunk.toString();
        });

        spawnedProcess.stderr.on("data", chunk => {
            errorData += chunk.toString();
        });

        spawnedProcess.on("close", function(code) {
            if (code > 0) {
                return reject(new Error(`${errorData} (Failed Instruction: ${instruction})`));
            }

            resolve(data);
        });

        spawnedProcess.on("error", function(err) {
            reject(err);
        });
    });
}

// example usage
async function run() {
    await spawn("echo hello");
    await spawn("echo hello", {}, true);
    await spawn("echo hello", { cwd: "/" });
}

run();