How can I filter JSon File using NodeJs streams

Question

I am using Node.JS streams to parse json file and plot it on map. I have two files : Front End where I plot the data

'/points.geojson'

Server side: 1- I load external file (output.json) .

var fs = require('fs');

app.get('/points.json', function(req, res) {
  res.setHeader('Content-Type', 'application/json');
  fs.createReadStream(__dirname + '/output.json').pipe(res);
});

2- Now I'd like to filter this data

I have three choices to do that

1- On Client-side : I am not sure if should I use Angular-JS Filtering, or pure JS using this library Filter.JS

I have a very huge file (100mb). So, let's consider this example:

 [
        {'name': 'Apple', 'colour': 'Red'},
        {'name': 'Orange', 'colour': 'Orange'},
        {'name': 'Banana', 'colour': 'Yellow'}]

http://jsfiddle.net/65Pyj/

let's say that I want to select all points with green color

var output = arr.filter(function(x){return x.colour =="green"}); 

2- On Server-side My problem that I wanna filter json data on the server side, so I don't load and re-load data (as it's big). I should filter the data on Node.Js.

When I checkbox "green", I wanna filter data. then re-load it again using the above-mentioned script.

I read that On express 3, we can use directly res.json({colour:'green'}) http://expressjs.com/api.html#res.json

But how can I make it listen to an event (checking box) on the client-side, then reload the jsonfile.

Any ideas how can I do that ? I am using Node.Js/express

Answer 1

For filtering server-side without loading and parsing the entire file at once, you might look into these modules which can parse JSON streams (files or otherwise):

• JSONStream
• oboejs
• clarinet -- this has a much lower level interface than the above two