Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What makes node.js SlowBuffers "slow"?

I'm using node.js to serve some PNG images that are stored in an SQLite database as binary BLOBs. These images are small, on average 9500 bytes.

I'm using the sqlite3 npm package, which appears to return binary BLOB objects as SlowBuffers. My node.js service holds these SlowBuffers in memory to mitigate IO latency, serving them like this:

response.send(slowBuffer);

It appears that SlowBuffer has an interface similar to Buffer; converting to Buffer is trivial:

var f = function(slowBuffer) {
  var buffer = new Buffer(slowBuffer.length);
  slowBuffer.copy(buffer);
  return buffer;
}

Should I convert these SlowBuffers to Buffers?

Help me understand why they are called "slow" buffers.

like image 507
Jacob Marble Avatar asked Dec 23 '12 06:12

Jacob Marble


People also ask

Why is NodeJS so slow?

Node. js programs can be slow due to a CPU/IO-bound operation, such as a database query or slow API call. For most Node. js applications, data fetching is done via an API request and a response is returned.

Why node js is no buffering?

No Buffering − Node. js applications never buffer any data. These applications simply output the data in chunks. Pure JavaScript does not handle straight binary data very well, though JavaScript is Unicode friendly.

What does buffer do in node JS?

Buffers in Node. js is used to perform operations on raw binary data. Generally, Buffer refers to the particular memory location in memory. Buffer and array have some similarities, but the difference is array can be any type, and it can be resizable. Buffers only deal with binary data, and it can not be resizable.


1 Answers

If you would read the posts:

  1. https://groups.google.com/forum/?fromgroups=#!topic/nodejs-dev/jd52ZsVSZNo
  2. https://groups.google.com/forum/?fromgroups=#!topic/nodejs/s1dnFbb-Rj8

Node provides two types of buffer objects. Buffer is a native Javascript data structure; SlowBuffer is implemented by a C++ module. Using C++ modules from the native Javascript environment costs extra CPU time, hence the "slow". Buffer objects are backed by SlowBuffer objects, but the contents can be read/written directly from Javascript for better performance.

Any Buffer object larger than 8 KB is backed by a single SlowBuffer object. Multiple Buffer objects smaller than 8 KB can be backed by a single SlowBuffer object. When many Buffer objects smaller than 8 KB exist in memory (backed by a single SlowBuffer), the C++ module penalty can be very high if you were to instead use a SlowBuffer for each one. Small Buffers are often used in large quantities.

This class is primarily for internal use means that if you want to manage buffers on the server on your own, then use SlowBuffer (to use in smaller chunks you have to partition SlowBuffer yourself). Unless you want that minute level control in handling buffers, you should be fine with using Buffer objects.

like image 116
user568109 Avatar answered Oct 01 '22 07:10

user568109