Writing to a JSON file (Using Node.js to work on a Localhost server)

Question

So I am using C3.js, a chart library built on top of the widely used D3.js visualization library. So as a part of the webpage that I am creating with a chart, I want to include a user interaction in which the user can input values for a new object that can be charted along with current objects that are already charted via a form on the html page. I'm using JSON data as the file format that's being read into the chart. I want to know how I can write to/update this file from the javascript code so that the data now includes the new object. As I am a novice with web-based technologies I'm not too sure as to how data gets updated on servers. The server is a localhost created by a file, app.js, which creates a node server.

var express = require('express');
var http = require('http');
var app = express();


app.use(express.static(__dirname + '/public'));

server = http.createServer(app);
server.listen('14000');

I included the above code for my app.js as a reference if that makes any difference. The file format for the json file is as shown below...

[
      {name: "Lane Warner",age: 27,height: 231,weight: 120},
      {name: "Hickman Bishop",age: 29,height: 125,weight: 180},
      {name: "Tracy Sheppard",age: 30,height: 200,weight: 155},
      {name: "Madeleine Spence",age: 30,height: 179,weight: 112},
      {name: "Alicia Beasley",age: 36,height: 300,weight: 200},
      {name: "Bryant Fitzpatrick",age: 23,height: 321,weight: 250},
      {name: "Stevenson Mcdonald",age: 30,height: 155,weight: 199},
      {name: "Hannah Ratliff",age: 21,height: 189,weight: 136},
      {name: "Alexandra Williamson",age: 39,height: 258,weight: 123}
]

Answer 1

You can use the fs.readFile module to read from a file, and JSON.parse to parse the JSON.

json-util.js:

var fs = require('fs');

module.exports.loadJson = function (file, callback) {
  fs.readFile('file.json', { encoding: 'utf8' }, function (err, data) {
    if (err) return callback(err); // file reading error
    try {
      // parse and return json to callback
      var json = JSON.parse(data);
      callback(null, json);
    } catch (ex) {
      // catch JSON parsing errors so your app doesn't crash
      callback(ex);
    }
  });
};

If you want to read the json file synchronously, you can also use json = require('file.json'). Note that require() is relative to the current file, and fs.readFile() is relative to the cwd.

Conversely, you can re-write the JSON to the file using JSON.stringify and fs.writeFile:

module.exports.writeJson = function (file, json, callback) {
  fs.writeFile(file, JSON.stringify(json), callback);
};

Also keep in mind any concurrency issues such as where two requests try to modify the same file at the same time. A timeline like this would be bad:

1. Request A reads the JSON.
2. Request B reads the JSON.
3. Request A adds an item to the array and re-writes the JSON.
4. Request B adds an item to the array and re-writes the JSON.

In this case, the addition request A made will be completely erased by request B. To avoid this, it would be best to always only have one copy of the JSON in memory, and re-write it every time you modify it. I think this is beyond the scope of your question, but wanted to point it out.

If you didn't care about concurrency, you could have an app like this:

app.js:

var express = require('express');
var http = require('http');
var app = express();

// if you're using Express 4.x, bodyParser is no longer built-in
var bodyParser = require('body-parser');

// here we import the file which has our loadJson and writeJson functions
var jsonUtil = require('./json-util.js');

app.use(express.static(__dirname + '/public'));
app.use(bodyParser.json());

app.post('/additem/:id', function (req, res, next) {
  // turn ID into a file name, and be careful not to expose security holes
  var filename = idToFileName(req.params.id);
  jsonUtil.loadJson(filename, function (err, json) {

    // TODO: make sure you handle errors
    // if err is not null, you can either consider it an error, or
    // you could simply say json = [] and start a new file

    // should also do validation checks like if(json instanceof Array) and
    // verify that req.body exists and is properly formatted, etc

    json.push(req.body); // push the object from the request body into the array

    // re-save the file
    jsonUtil.writeJson(filename, json, function (err) {
      if (err) next(err);
      else res.send(200);
    });
  });
});

server = http.createServer(app);
server.listen('14000');

Then, a request which looks something like:

POST /additem/1
Content-Type: application/json

{ "name": "Alexandra Williamson", "age": 39, "height": 258, "weight": 123 }

Will append to an array in which ever json file id = 1 maps to.