Creating a typed array from a byte array stored as a node Buffer

Question

From the node docs regarding the creation of typed arrays from Buffers:

The buffer's memory is interpreted as an array, not a byte array. That is, new Uint32Array(new Buffer([1,2,3,4])) creates a 4-element Uint32Array with elements [1,2,3,4], not an Uint32Array with a single element [0x1020304] or [0x4030201].

This contrasts to plain javascript, where creating a typed array view from an ArrayBuffer uses the ArrayBuffer's memory as bytes (like a reinterpret_cast in C++). I need this behavior in node when operating on node Buffers.

I could convert the Buffer to an ArrayBuffer, but this is too slow for my application. (I've tried many methods -- but they're all O(n) time.) (Edit: the fastest method I've found is this, which is a single memmove op and pretty fast, but still has at least momentary 2x memory consumption until the reference to the original Buffer is released.)

Is there any (fast/O(1)) way to get a typed array from a Buffer, using the Buffer's contents as bytes instead of elements? (The needed typed array element size is >1 byte, needless to say.)

Answer 1

As far as I know, it is not possible to do this without making a copy of the data in memory. Even your example new Uint32Array(new Buffer([1,2,3,4])) internally does this (meaning it's not O(1)).

Note that typed arrays are just views of an ArrayBuffer (not Buffer, that's not possible). new Uint32Array(array) creates an ArrayBuffer of 4 * array.length bytes. You can access it with uint32Array.buffer. The constructor treats your Buffer no different than a normal Array.

The best solution I know is the one you already found.

Another issue with using Uint32Array for what you try to do is that it depends on platform byte order. You could either iterate over the Buffer like this or use DataView if you want to be safe.

Answer 2

As of node 4.0, Buffers are Uint8Arrays and new views can be constructed on them directly:

var b = new Buffer([1,2,3,4]);
new Uint32Array(b.buffer, b.byteOffset);

This mini-example assumes that b is larger than ~4096 (the threshold for a Buffer owning its own chunk of memory vs. sharing a larger chunk). See Convert a binary NodeJS Buffer to JavaScript ArrayBuffer for more info -- generally you should call b.slice() to operate on only the memory you own.