Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How is Node JS Buffer data stored behind the scenes?

According to the Node JS Buffer Documentation, "A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap". No further information is given.

The question is how the data is stored in RAM. Does the node JS buffer use a special way of allocating space on the heap? Is that subject to the same garbage collection as V8's heap? Am I safe to assume that any change to the data in a buffer actually changes the data in RAM, and that no residual remnants of the data is left for snoopers?

Sorry about the very broad question, but I can't seem to find any material on how this actually works. The reason I am asking is because I want to make sure that the variables I use in my application don't stick around in memory for longer than they need to.

Docs: https://nodejs.org/api/buffer.html#buffer_class_buffer

Cheers!

like image 977
Axel Ericsson Avatar asked Jul 25 '15 00:07

Axel Ericsson


People also ask

How does the buffer store data in node JS?

A buffer is a space in memory (typically RAM) that stores binary data. In Node. js, we can access these spaces of memory with the built-in Buffer class. Buffers store a sequence of integers, similar to an array in JavaScript.

How node JS works behind the scene?

Event-loop is what allows Node. js applications to run non-blocking asynchronous I/O based operations i.e, all the asynchronous code is managed and executed within the event-loop and before that, we can use our synchronous code which is in this case known as Top-Level code.

How does node js store data locally?

Storing your Node. js application's configuration data is quite simple - every object in JavaScript can be easily rendered as JSON, which in turn is just string data that can be sent or saved any way you'd like. The simplest way to do this involves the built-in JSON. parse() and JSON.


1 Answers

The nodejs source code for implementing buffers is http://github.com/joyent/node/blob/master/lib/buffer.js and http://github.com/joyent/node/blob/master/src/node_buffer.cc. If you really want to know the details of how it works, you can study the actual code.

As for your questions...

The question is how the data is stored in RAM. Does the node JS buffer use a special way of allocating space on the heap?

Per the source code, there are both heap allocations and a memory pool involved in memory allocated to buffer objects. When each is used depends upon details of how it is used.

Is that subject to the same garbage collection as V8's heap?

Yes, garbage collection works for Buffer objects. It is possible for objects that are implemented with native code to participate in garbage collection if they follow a strict set of rules in their implementation.

Am I safe to assume that any change to the data in a buffer actually changes the data in RAM, and that no residual remnants of the data is left for snoopers?

Yes, when you change data in a buffer, it does actually change data in RAM (there is no other place to store the change besides RAM unless it was only stored on disk which is not the case).

As for "no residual remnants of the data left for snoopers", that is hard to say. It is very common in programming that when heap elements or pooled elements grow or shrink that their data may be copied into a different piece of RAM and the old, now freed or recycled block of RAM may still have a copy of some or all of the original data. Unless specifically cleared, newly allocated RAM may very well have been used for something else before and may still contain information from that previous use.

On the other hand, writing to a Buffer object writes directly to the RAM that is allocated/assigned to that Buffer object (as long as you don't cause its size to change) so if you want to decrease the odds that your data would be left around in RAM, you can overwrite the data in your Buffer object when you are done with it.

like image 85
jfriend00 Avatar answered Sep 28 '22 18:09

jfriend00