Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NodeJS ReadStream not reading bufferSize bytes at a time

I have a code where the NodeJS server reads a file and streams it to response, it looks like:

var fStream = fs.createReadStream(filePath, {'bufferSize': 128 * 1024});
fStream.pipe(response);

The issue is, Node reads the file exactly 40960 bytes a time. However, my app would be much more efficient (due to reasons not applicable to this question), if it reads 131072 (128 * 1024) bytes at a time.

Is there a way to force Node to read 128 * 1024 bytes at a time?

like image 257
Tarandeep Gill Avatar asked Aug 02 '12 19:08

Tarandeep Gill


1 Answers

The accepted answer is wrong. You can force Node to read (128*1024) bytes at a time using the highWaterMark option.

var fStream = fs.createReadStream('/foo/bar', { highWaterMark: 128 * 1024 });

The Documentation specifically states that 'the amount of data potentially buffered depends on the highWaterMark option passed into the streams constructor. For normal streams, the highWaterMark option specifies a total number of bytes. For streams operating in object mode, the highWaterMark specifies a total number of objects.'

Also, see this. The default buffer size is 64 KiloBytes

like image 51
Shwetabh Shekhar Avatar answered Oct 14 '22 02:10

Shwetabh Shekhar