Write objects into file with Node.js

Question

I've searched all over stackoverflow / google for this, but can't seem to figure it out.

I'm scraping social media links of a given URL page, and the function returns an object with a list of URLs.

When I try to write this data into a different file, it outputs to the file as [object Object] instead of the expected: [ 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'] as it does when I console.log() the results.

This is my sad attempt to read and write a file in Node, trying to read each line(the url) and input through a function call request(line, gotHTML):

fs.readFileSync('./urls.txt').toString().split('\n').forEach(function (line){     console.log(line);      var obj = request(line, gotHTML);      console.log(obj);      fs.writeFileSync('./data.json', obj , 'utf-8');  });    

for reference -- the gotHTML function:

function gotHTML(err, resp, html){      var social_ids = [];       if(err){         return console.log(err);      } else if (resp.statusCode === 200){          var parsedHTML = $.load(html);           parsedHTML('a').map(function(i, link){             var href = $(link).attr('href');             for(var i=0; i<socialurls.length; i++){                 if(socialurls[i].test(href) && social_ids.indexOf(href) < 0 ) {                     social_ids.push(href);                  };              };          })     };      return social_ids; }; 

Answer 1

Building on what deb2fast said I would also pass in a couple of extra parameters to JSON.stringify() to get it to pretty format:

fs.writeFileSync('./data.json', JSON.stringify(obj, null, 2) , 'utf-8'); 

The second param is an optional replacer function which you don't need in this case so null works.

The third param is the number of spaces to use for indentation. 2 and 4 seem to be popular choices.

Answer 2

obj is an array in your example.

fs.writeFileSync(filename, data, [options]) requires either String or Buffer in the data parameter. see docs.

Try to write the array in a string format:

// writes 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks' fs.writeFileSync('./data.json', obj.join(',') , 'utf-8');  

Or:

// writes ['https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'] var util = require('util'); fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8'); 

edit: The reason you see the array in your example is because node's implementation of console.log doesn't just call toString, it calls util.format see console.js source