I have the url to a possibly large (100+ Mb) file, how do I save it in a local directory using fetch?
I looked around but there don't seem to be a lot of resources/tutorials on how to do this.
Thank you!
writeFile("/tmp/test", message, function (err) { if (err) { return console. log(err); } console. log("The file was saved!"); }); }); Hope this works for you.
Fetch is already available as an experimental feature in Node v17. If you're interested in trying it out before the main release, you'll need to first download and upgrade your Node. js version to 17.5.
Approach: First make the necessary JavaScript file, HTML file and CSS file. Then store the API URL in a variable (here api_url). Define a async function (here getapi()) and pass api_url in that function. Define a constant response and store the fetched data by await fetch() method.
Create Controller for download File It exports 2 functions: getListFiles() : read all files in uploads folder, return list of files' informationn (name, url) download() : receives file name as input parameter, then uses Express res. download API to transfer the file at path (directory + file name) as an 'attachment'.
Using the Fetch API you could write a function that could download from a URL like this:
const downloadFile = (async (url, path) => { const res = await fetch(url); const fileStream = fs.createWriteStream(path); await new Promise((resolve, reject) => { res.body.pipe(fileStream); res.body.on("error", reject); fileStream.on("finish", resolve); }); });
If you want to avoid explicitly making a Promise like in the other very fine answer, and are ok with building a buffer of the entire 100+ MB file, then you could do something simpler:
const fetch = require('node-fetch');
const {writeFile} = require('fs');
const {promisify} = require('util');
const writeFilePromise = promisify(writeFile);
function downloadFile(url, outputPath) {
return fetch(url)
.then(x => x.arrayBuffer())
.then(x => writeFilePromise(outputPath, Buffer.from(x)));
}
But the other answer will be more memory-efficient since it's piping the received data stream directly into a file without accumulating all of it in a Buffer.
const {createWriteStream} = require('fs');
const {pipeline} = require('stream/promises');
const fetch = require('node-fetch');
const downloadFile = async (url, path) => pipeline(
(await fetch(url)).body,
createWriteStream(path)
);
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With