Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

convert streamed buffers to utf8-string

Tags:

node.js

People also ask

Is node buffer a stream?

Streams work on a concept called buffer. A buffer is a temporary memory that a stream takes to hold some data until it is consumed. In a stream, the buffer size is decided by the highWatermark property on the stream instance which is a number denoting the size of the buffer in bytes.

How do I convert a buffer to a string?

Buffers have a toString() method that you can use to convert the buffer to a string. By default, toString() converts the buffer to a string using UTF8 encoding. For example, if you create a buffer from a string using Buffer. from() , the toString() function gives you the original string back.

Which buffer class methods returns an uninitialized buffer?

Answers related to “which buffer class methods returns an uninitialized buffer” Create buffers from strings using the Buffer. from() function. Like toString(), you can pass an encoding argument to Buffer.

What are Nodejs buffers?

What Are Buffers? The Buffer class in Node. js is designed to handle raw binary data. Each buffer corresponds to some raw memory allocated outside V8. Buffers act somewhat like arrays of integers, but aren't resizable and have a whole bunch of methods specifically for binary data.


Single Buffer

If you have a single Buffer you can use its toString method that will convert all or part of the binary contents to a string using a specific encoding. It defaults to utf8 if you don't provide a parameter, but I've explicitly set the encoding in this example.

var req = http.request(reqOptions, function(res) {
    ...

    res.on('data', function(chunk) {
        var textChunk = chunk.toString('utf8');
        // process utf8 text chunk
    });
});

Streamed Buffers

If you have streamed buffers like in the question above where the first byte of a multi-byte UTF8-character may be contained in the first Buffer (chunk) and the second byte in the second Buffer then you should use a StringDecoder. :

var StringDecoder = require('string_decoder').StringDecoder;

var req = http.request(reqOptions, function(res) {
    ...
    var decoder = new StringDecoder('utf8');

    res.on('data', function(chunk) {
        var textChunk = decoder.write(chunk);
        // process utf8 text chunk
    });
});

This way bytes of incomplete characters are buffered by the StringDecoder until all required bytes were written to the decoder.