What is a compact way to save a float32array to disk on node.js?

Question

JSON.stringify is obviously not space-efficient. What is the most elegant way to serialize and store a float32array using Node.js?

EDIT: People are closing the question for reasons such as being "opinion based" and "lack of an understanding of the problem". I seriously believe the first one was a missclick. For the second one, maybe this makes it more clear:

var fs = require("fs");
var len = 1000*1000*10;
var big_array = new Float32Array(len);
for (var i=0; i<len; ++i)
    big_array[i] = Math.random();

// OBVIOUSLY NOT SPACE EFFICIENT \/
fs.writeFileSync("big_array.json",JSON.stringify(big_array));

It is not space efficient because you are representing numbers as strings, so an 8 bytes float will be using as much as ~20 utf8 chars, which is a waste. The question is: how to store the array in a space-efficient manner?

Answer 1

Finally I managed to write float32array to disk with nodejs and retrieve them on the browser, and I hope it will help you.

Write Float32Array to binary file in NodeJS

    var fs = require('fs');
    var wstream = fs.createWriteStream('data.dat');

    var data = new Float32Array([1.1,2.2,3.3,4.4,5.5]);

    //prepare the length of the buffer to 4 bytes per float
    var buffer = new Buffer(data.length*4);


    for(var i = 0; i < data.length; i++){
        //write the float in Little-Endian and move the offset
        buffer.writeFloatLE(data[i], i*4);
    }

    wstream.write(buffer);
    wstream.end();

Read the file and convert it to a Float32Array on a Browser

    var urlToFloatFile = 'data.dat';
    var request = new XMLHttpRequest();
    request.open('GET', urlToFloatFile, true);

    //specify the response type as arraybuffer
    request.responseType = 'arraybuffer';

    request.onload = function (msg) { 
        var yourFloatData = new Float32Array(this.response);
        console.log(yourFloatData);
    };
    request.send();

Thanks to @ben_a_adams from WebGL Dev List GGroup https://groups.google.com/forum/#!topic/webgl-dev-list/EbGUi_iSEx8 for the client side code

I've create a simple test to test roughly how much space a JSON serialization of a float array differs from a binary representation and the results are:

• 2.000.000 floating point values

• 7.8MB on a binary file

• 38.5MB on a JSON file

• 17.5 on a Gzipped JSON file

Answer 2

There is actually a much simpler version possible

let fs = require('fs')
let data = [150, 180]
fs.writeFileSync('mydata', new Buffer(new Uint32Array(data).buffer))
fs.readFile('mydata', (err, buf) => {
    let restoredData = new Uint32Array(buf.buffer, buf.offset, buf.byteLength/4)
    console.log(data[1])
    console.log(restoredData[1])
});