Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to create a readable stream from a remote url in nodejs?

on nodejs documentation, the streams section says I can do fs.createReadStream(url || path). But, when I actually do that It tells me Error: ENOENT: no such file or directory. I just want to pipe the video from a readable to a writable stream, But I'm stuck on creating a readable one.

my code:

const express = require('express')
const fs = require('fs')
const url = 'https://www.example.com/path/to/mp4Video.mp4'
const port = 3000

app.get('/video', (req, res) => {
    const readable = fs.createReadStream(url)
})
app.listen(port, () => {
    console.log('listening on port ' + port)
})

the ERROR:

listening on port 3000
events.js:291
      throw er; // Unhandled 'error' event
      ^

Error: ENOENT: no such file or directory, open 'https://www.example.com/path/to/mp4Video.mp4'
Emitted 'error' event on ReadStream instance at:
    at internal/fs/streams.js:136:12
    at FSReqCallback.oncomplete (fs.js:156:23) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'https://www.example.com/path/to/mp4Video.mp4'
}

PS: https://www.example.com/path/to/mp4Video.mp4 IS NOT THE ACTUAL URL

like image 689
Martin Wittick Avatar asked Jan 31 '21 06:01

Martin Wittick


1 Answers

fs.createReadStream() does not work with http URLs only file:// URLs or filename paths. Unfortunately, this is not described in the fs doc, but if you got look at the source code for fs.createReadStream() and follow what it calls you can find that it ends up calling fileURULtoPath(url) which will throw if it's not a file: URL.

function fileURLToPath(path) {
  if (typeof path === 'string')
    path = new URL(path);
  else if (!isURLInstance(path))
    throw new ERR_INVALID_ARG_TYPE('path', ['string', 'URL'], path);
  if (path.protocol !== 'file:')
    throw new ERR_INVALID_URL_SCHEME('file');
  return isWindows ? getPathFromURLWin32(path) : getPathFromURLPosix(path);
}

It would suggest using the got() library to get yourself a readstream from a URL:

const got = require('got');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get('/video', (req, res) => {
    got.stream(mp4Url).pipe(res);
});

More examples described in this article: How to stream file downloads in Nodejs with Got.


You can also use the plain http/https modules to get the readstream, but I find got() to be generally useful at a higher level for lots of http request things so that's what I use. But, here's code with the https module.

const https = require('https');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get("/", (req, res) => {
    https.get(mp4Url, (stream) => {
        stream.pipe(res);
    });
});

More advanced error handling could be added to both cases.

like image 180
jfriend00 Avatar answered Nov 15 '22 10:11

jfriend00