Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I read the contents of a Node.js stream into a string variable?

People also ask

How do I read a node readable stream?

read() Method. The readable. read() method is an inbuilt application programming interface of Stream module which is used to read the data out of the internal buffer. It returns data as a buffer object if no encoding is being specified or if the stream is working in object mode.

How do I get data from a readable stream?

To retrieve data from a JavaScript ReadableStream object, we need to call a conversion method to convert the ReadableStream to the actual data we can use. to make a GET request to https://httpbin.org/ip with fetch . fetch returns a promise that resolves to a ReadableStream object which we assigned to response .

What is read stream in node JS?

Streams are objects that let you read data from a source or write data to a destination in continuous fashion. In Node.js, there are four types of streams − Readable − Stream which is used for read operation.


Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.

function streamToString (stream) {
  const chunks = [];
  return new Promise((resolve, reject) => {
    stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
    stream.on('error', (err) => reject(err));
    stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
  })
}

const result = await streamToString(stream)

None of the above worked for me. I needed to use the Buffer object:

  const chunks = [];

  readStream.on("data", function (chunk) {
    chunks.push(chunk);
  });

  // Send the buffer or you can put it into a var
  readStream.on("end", function () {
    res.send(Buffer.concat(chunks));
  });

Hope this is more useful than the above answer:

var string = '';
stream.on('data',function(data){
  string += data.toString();
  console.log('stream data ' + part);
});

stream.on('end',function(){
  console.log('final output ' + string);
});

Note that string concatenation is not the most efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency).

Also, this code may produce unpredictable failures for non-ASCII text (it assumes that every character fits in a byte), but perhaps you do not care about that, either.


(This answer is from years ago, when it was the best answer. There is now a better answer below this. I haven't kept up with node.js, and I cannot delete this answer because it is marked "correct on this question". If you are thinking of down clicking, what do you want me to do?)

The key is to use the data and end events of a Readable Stream. Listen to these events:

stream.on('data', (chunk) => { ... });
stream.on('end', () => { ... });

When you receive the data event, add the new chunk of data to a Buffer created to collect the data.

When you receive the end event, convert the completed Buffer into a string, if necessary. Then do what you need to do with it.


What do you think about this ?

async function streamToString(stream) {
    // lets have a ReadableStream as a stream variable
    const chunks = [];

    for await (const chunk of stream) {
        chunks.push(Buffer.from(chunk));
    }

    return Buffer.concat(chunks).toString("utf-8");
}


I'm using usually this simple function to transform a stream into a string:

function streamToString(stream, cb) {
  const chunks = [];
  stream.on('data', (chunk) => {
    chunks.push(chunk.toString());
  });
  stream.on('end', () => {
    cb(chunks.join(''));
  });
}

Usage example:

let stream = fs.createReadStream('./myFile.foo');
streamToString(stream, (data) => {
  console.log(data);  // data is now my string variable
});

And yet another one for strings using promises:

function getStream(stream) {
  return new Promise(resolve => {
    const chunks = [];

    # Buffer.from is required if chunk is a String, see comments
    stream.on("data", chunk => chunks.push(Buffer.from(chunk)));
    stream.on("end", () => resolve(Buffer.concat(chunks).toString()));
  });
}

Usage:

const stream = fs.createReadStream(__filename);
getStream(stream).then(r=>console.log(r));

remove the .toString() to use with binary Data if required.

update: @AndreiLED correctly pointed out this has problems with strings. I couldn't get a stream returning strings with the version of node I have, but the api notes this is possible.