How do I download a website and write it to a file using Restler?

Question

I would like to download a website (html) and write it to an .html file with node and restler.

https://github.com/danwrong/Restler/

Their initial example is already halfway there:

var sys = require('util'),
    rest = require('./restler');

rest.get('http://google.com').on('complete', function(result) {
  if (result instanceof Error) {
    sys.puts('Error: ' + result.message);
    this.retry(5000); // try again after 5 sec
  } else {
    sys.puts(result);
  }
});

Instead of sys.puts(result);, I would need to save it to a file.

I am confused if I need a Buffer, or if I can write it directly to file.

Answer 1

You can simply use fs.writeFile in node:

fs.writeFile(__dirname + '/file.txt', result, function(err) {
    if (err) throw err;
    console.log('It\'s saved!');
});

Or streamed more recommended approach, that can handle very large files and is very memory efficient:

// create write stream
var file = fs.createWriteStream(__dirname + '/file.txt');

// make http request
http.get('http://example.com/', function(res) {
    // pipe response into file
    res.pipe(file);
    // once done
    file.on('finish', function() {
        // close write stream
        file.close(function(err) {
            console.log('done');
        });
    });
});