How to pass a Buffer as argument of fs.createReadStream

Question

According to docs https://nodejs.org/api/fs.html#fs_fs_createreadstream_path_options

fs.createReadStream() can accept Buffer as first argument

my node code:

var _ = require('lodash')
var faker = require('faker')
var http = require('http')
var fs = require('fs')
var xlsx = require('node-xlsx')

var gg = _.range(10).map((item) => {
  return _.range(10).map((item) => {
    return faker.name.findName()
  })
})

http.createServer(function(req, res) {
  var buf = xlsx.build([{
    name: 'sheet1',
    data: gg
  }])
  fs.createReadStream(buf, 'binary').pipe(res)

}).listen(9090)

but I get this error:

events.js:160
  throw er; // Unhandled 'error' event
  ^

Error: Path must be a string without null bytes
at nullCheck (fs.js:135:14)
at Object.fs.open (fs.js:627:8)
at ReadStream.open (fs.js:1951:6)
at new ReadStream (fs.js:1938:10)
at Object.fs.createReadStream (fs.js:1885:10)
at Server.<anonymous> (/Users/xpg/project/test/index.js:18:6)
at emitTwo (events.js:106:13)
at Server.emit (events.js:191:7)
at HTTPParser.parserOnIncoming [as onIncoming] (_http_server.js:546:12)
at HTTPParser.parserOnHeadersComplete (_http_common.js:99:23)

I just want to know that if I want to pass a Buffer as the path argument, what is the options I should provide, passing 'binary' doesn't work.

I try it with both Node 6.11.0 and Node 8.4.0

Answer 1

The first argument to fs.createReadStream() must be the file path. You can apparently pass the path in a Buffer object, but it still must be an acceptable OS path when the Buffer is converted to a string.

It appears you are trying to pass the file content to fs.createReadStream(). That is not how that API works. If you look into the code for fs.createReadStream() it is completely clear in the code that it is going to call fs.open() and pass the first argument from fs.createReadStream() as the file path for fs.open().

If what you're trying to do is to create a readable stream from a buffer (no file involved), then you need to do that a different way. You can see how to do that here in this answer: Converting a Buffer into a ReadableStream in Node.js.

Conceptually, you just create a readable stream object, push the data you already have into it from your Buffer, push a null to signify the end of the stream and create a noop _read() method and you're done. You can then use that readable stream with any other code that expects to read it.

Answer 2

Creating a Readable Stream From Buffer.

You can easily create a Readable Stream from a Buffer, however using fs.createReadStream() does indeed require first writing it to a file path.

Using stream.Duplex()

• No need to write a local file in order to get a readable stream
• Saves I/O and speeds things up.
• Works everywhere fs.createReadStream() is desired.
• Great for writing to cloud services where local storage my not be feasible.

Example:

const {Duplex} = require('stream'); // Native Node Module 

function bufferToStream(myBuffer) {
    let tmp = new Duplex();
    tmp.push(myBuffer);
    tmp.push(null);
    return tmp;
}

const myReadableStream = bufferToStream(your_buffer);

// use myReadableStream anywhere you would use a stream 
// created using fs.createReadStream('some_path.ext');
// For really large streams, you may want to pipe the buffer into the Duplex.

Answer 3

@jfriend00 has already provided a very clear explanation on this issue. If a Buffer object is passed as argument to fs.createReadStream(), it should indicate the file path, not file content. As @Littlee asked in comment, here is an example code:

var express = require('express');
var router = express.Router();
var fs = require('fs')

router.get('/test', function(req, res) {
  var buf = Buffer.from('./test.html');
  fs.createReadStream(buf).pipe(res);
});

Please note the Buffer buf indicates a file path ("./test.html"), not the file test.html's content.