Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Read json file content with require vs fs.readFile

Suppose that for every response from an API, i need to map the value from the response to an existing json file in my web application and display the value from the json. What are the better approach in this case to read the json file? require or fs.readfile. Note that there might be thousands of request comes in at a same time.

Note that I do not expect there is any changes to the file during runtime.

request(options, function(error, response, body) {
   // compare response identifier value with json file in node
   // if identifier value exist in the json file
   // return the corresponding value in json file instead
});
like image 697
vincentsty Avatar asked Feb 14 '16 06:02

vincentsty


People also ask

Can FS read json file?

Read and Parse a JSON File From DiskUse the Fs. readFile to read the content of a given file. Related to a JSON file, you'll then have the JSON content as a string. You can use the package object like you would use any other JavaScript object.

What does the readFile () method require?

readFile() method is used to read the file. This method read the entire file into buffer. To load the fs module, we use require() method. It Asynchronously reads the entire contents of a file.

What is the difference between FS readFile and FS readFileSync?

In fs. readFile() method, we can read a file in a non-blocking asynchronous way, but in fs. readFileSync() method, we can read files in a synchronous way, i.e. we are telling node.

Does readFile return anything?

The ReadFile function returns when one of the following conditions occur: The number of bytes requested is read. A write operation completes on the write end of the pipe. An asynchronous handle is being used and the read is occurring asynchronously.


4 Answers

I suppose you'll JSON.parse the json file for the comparison, in that case, require is better because it'll parse the file right away and it's sync:

var obj = require('./myjson'); // no need to add the .json extension

If you have thousands of request using that file, require it once outside your request handler and that's it:

var myObj = require('./myjson');
request(options, function(error, response, body) {
   // myObj is accessible here and is a nice JavaScript object
   var value = myObj.someValue;

   // compare response identifier value with json file in node
   // if identifier value exist in the json file
   // return the corresponding value in json file instead
});
like image 83
Shanoor Avatar answered Oct 13 '22 22:10

Shanoor


There are two versions for fs.readFile, and they are

Asynchronous version

require('fs').readFile('path/test.json', 'utf8', function (err, data) {
    if (err) 
       // error handling

    var obj = JSON.parse(data);
});

Synchronous version

var json = JSON.parse(require('fs').readFileSync('path/test.json', 'utf8'));

To use require to parse json file as below

var json = require('path/test.json');

But, note that

  • require is synchronous and only reads the file once, following calls return the result from cache

  • If your file does not have a .json extension, require will not treat the contents of the file as JSON.

like image 34
zangw Avatar answered Oct 13 '22 22:10

zangw


Since no one ever cared to write a benchmark, and I had a feeling that require works faster, I made one myself.

I compared fs.readFile (promisified version) vs require (without cache) vs fs.readFileSync.

You can see benchmark here and results here.

For 1000 iterations, it looks like this:

require: 835.308ms
readFileSync: 666.151ms
readFileAsync: 1178.361ms

So what should you use? The answer is not so simple.

  1. Use require when you need to cache object forever. And better use Object.freeze to avoid mutating it in application.
  2. Use readFileSync in unit tests or on blocking application startup - it is fastest.
  3. Use readFile or promisified version when application is running and you don't wanna block event loop.
like image 14
Jehy Avatar answered Oct 13 '22 22:10

Jehy


Use node-fixtures if dealing with JSON fixtures in your tests.

The project will look for a directory named fixtures which must be child of your test directory in order to load all the fixtures (*.js or *.json files):

// test/fixtures/users.json
{
  "dearwish": {
    "name": "David",
    "gender": "male"
  },
  "innaro": {
    "name": "Inna",
    "gender": "female"
  }
}
// test/users.test.js
var fx = require('node-fixtures');
fx.users.dearwish.name; // => "David" 
like image 3
Stéphane Bruckert Avatar answered Oct 13 '22 21:10

Stéphane Bruckert