Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Node.js ReadableStream: what is chunk size?

Tags:

node.js

Let's look at the following code snippet:

const fs = require('fs');
const server = require('http').createServer();

server.on('request', (req, res) => {
  const src = fs.createReadStream('./big.file');
  src.pipe(res);
});

server.listen(8000);

fs.createReadStream divides the content of big.file to chunks and reads the file "chunk after chunk".

My question is what is the size of every chunk, where it's defined and how I can change the chunk size?

like image 733
CrazySynthax Avatar asked Mar 05 '19 23:03

CrazySynthax


People also ask

What is chunk in node JS?

A chunk is a fragment of the data that is sent by the client to server all chunks concepts to each other to make a buffer of the stream then the buffer is converted into meaningful data.

What does createReadStream return?

Return Value: This method returns the fs. ReadStream object.

What is createReadStream in node JS?

createReadStream() allows you to open up a readable stream in a very simple manner. All you have to do is pass the path of the file to start streaming in. It turns out that the response (as well as the request) objects are streams. So we will use this fact to create a http server that streams the files to the client.

What is stream PassThrough ()?

PassThrough. This Stream is a trivial implementation of a Transform stream that simply passes the input bytes across to the output. This is mainly for testing and some other trivial use cases. Here is an example of Passthrough Stream where it is piping from readable stream to writable stream.


2 Answers

here is how to change the chunk size

var rs = fs.createReadStream('/foo/bar', { highWaterMark: 128 * 1024 });
like image 197
ALPHA Avatar answered Sep 30 '22 18:09

ALPHA


Check the documentation for createReadStream here.

It takes in an options object as its second parameter.

highWaterMark is the option that you are looking for, and it defaults to 64 * 1024.

like image 26
tehp Avatar answered Sep 30 '22 18:09

tehp