nodejs json.stringify a 1gb object running out of memory

Question

I'm trying to json.stringify a 1 gb object so that I can write it to disk. I get FATAL ERROR: JS Allocation failed - process out of memory

What can I do to stringify successfully?

Answer 1

You can stringify bit by bit manually: if y is a key of x, then JSON.stringify(y) + ":" + JSON.stringify(x[y]) gives you one segment.

Using fs.appendFileSync, for example, you can use:

var output = "out.json";
fs.writeFileSync(output, "{");
var first = true;
for(var y in x) {
    if(x.hasOwnProperty(y)) {
        if(first) first = false;
        else fs.appendFileSync(output, ",");
        fs.appendFileSync(output, JSON.stringify(y) + ":" + JSON.stringify(x[y]))
    }
} 
fs.appendFileSync(output, "}");

You can also use a Write Stream