Script output is buffered into one message, despite separate echo statements?

Question

I have a shell script with three echo statements:

echo 'first message'

echo 'second message'

echo 'third message'

I then run this script in node and collect the output via this code:

var child = process.spawn('./test.sh');
child.stdout.on('data', data => {
   data = JSON.stringify(data.toString('utf8'));
   console.log(data);
});

But the singular output is "first message\nsecond message\nthird message\n", which is a problem. I expected three outputs, not one smushed together due to some form of buffering. And I can't just split on newlines, because the individual outputs may contain newlines.

Is there any way to distinguish the messages of individual echo statements? (or other output commands, i.e. printf, or anything that causes data to be written to stdout or stderror)

Edit: I have tried unbuffer and stdbuf, neither work for this, as simple testing can demonstrate. Here is an example of the stdbuf attempt, which I tried with a variety of different argument values, essentially all possible options.

 var child = process.spawn('stdbuf', ['-i0', '-o0', '-e0', './test.sh']);

To be clear, this problem happens when I run a python script from node, too, with just three simple print statements. So it's language-agnostic, it's not about bash scripting in particular. It's about successfully detecting the individual outputs of a script in any language on a unix-based system. If this is something C/C++ can do and I have to hook into that from node, I'm willing to go there. Any working solution is welcome.

---

Edit: I solved the problem for myself initially by piping the script's output to sed and using s/$/uniqueString to insert an identifier at the end of each individual output, then just splitting the received data on that identifier.

The answer I gave the bounty to will work on single-line outputs, but breaks on multi-line outputs. A mistake in my testing led me to think was not the case, but it is. The accepted answer is the better solution and will work on outputs of any size. But if you can't control the script and have to handle user-created scripts, then my sed solution is the only thing I've found that works. And it does work, quite well.

Answer 1

You can use the readline interface provided as part of the node APIs. More information here https://nodejs.org/api/readline.html#readline_event_line. You will use spawn as it is however pass the stdout to readline so that it can parse the lines. Not sure if this is what you intend to do. Here is some sample code:

var process = require('child_process');
const readline = require('readline');

var child = process.spawn('./test.sh');

// Use readline interface
const readlinebyline = readline.createInterface({ input: child.stdout });

// Called when a line is received
readlinebyline.on('line', (line) => {
    line = JSON.stringify(line.toString('utf8'));
    console.log(line);
});

Output:

"first message"
"second message"
"third message"

If you get an error like TypeError: input.on is not a function, make sure you have executing privileges on the test.sh script via chmod +x test.sh.