Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do I need await fs.createWriteStream in pipe method in node?

I am very confuse using pipe to handle write stream is sync or not, because I find a question about callback to handle completion of pipe

I just wanna ensure the write stream is done before do others like fs.rename, so I promisify it, code like:

(async function () {
  await promiseTempStream({oldPath, makeRegex, replaceFn, replaceObj, tempPath})
  await rename(tempPath, oldPath)
  function promiseTempStream({oldPath, makeRegex, replaceFn, replaceObj, tempPath}) {
  return new Promise((res, rej) => {
    const writable = fs.createWriteStream(tempPath)
    fs.createReadStream(oldPath, 'utf8')       
      .pipe(replaceStream(makeRegex ,replaceFn.bind(this, replaceObj), {maxMatchLen: 5000}))
    .pipe(writable)
    writable
      .on('error', (err) => {rej(err)})
      .on('finish', res)
    })
}
}())

It works, but I’m confused after read pipe doc, because it says 

By default, stream.end() is called on the destination Writable stream when the source Readable stream emits 'end', so that the destination is no longer writable.

So I only need

await fs.createReadStream(oldPath, 'utf8')
.pipe(replaceStream(makeRegex ,replaceFn.bind(this, replaceObj), {maxMatchLen: 5000}))
.pipe(fs.createWriteStream(tempPath))
await rename(tempPath, oldPath)

or just

fs.createReadStream(oldPath, 'utf8')
.pipe(replaceStream(makeRegex ,replaceFn.bind(this, replaceObj), {maxMatchLen: 5000}))
.pipe(fs.createWriteStream(tempPath))
await rename(tempPath, oldPath)

which is right way to do it? thank you very much

like image 414
Roy Avatar asked Oct 15 '17 06:10

Roy


People also ask

What is FS createWriteStream in node JS?

The function fs. createWriteStream() creates a writable stream in a very simple manner. After a call to fs. createWriteStream() with the filepath, you have a writeable stream to work with. It turns out that the response (as well as the request) objects are streams.

Is createReadStream asynchronous?

createReadStream() is an asynchronous operation that has completion events. But, because of the way it's been combined with reading from the file, you don't generally have to use it like it's asynchronous because it's asynchronous behavior is combined with the async reading from the file.

Are node streams asynchronous?

Streams are a built-in Node. js language feature that represent an asynchronous flow of data, and are a way to handle reading/writing files.

How does NodeJs pipe work?

pipe() Method. The readable. pipe() method in a Readable Stream is used to attach a Writable stream to the readable stream so that it consequently switches into flowing mode and then pushes all the data that it has to the attached Writable.


1 Answers

You need to wait for the finish event on the tempPath stream. So you could do something like

async function createTheFile() {
return new Promise<void>(resolve => {
    let a = replaceStream(makeRegex, replaceFn.bind(this, replaceObj), { maxMatchLen: 5000 });
    let b = fs.createWriteStream(tempPath);
    fs.createReadStream(oldPath, 'utf8').pipe(a).pipe(b);
    b.on('finish', resolve);
}
}

await createTheFile();
rename(tempPath, oldPath);

Basically here we have created a promise that resolves when we have completed writing into the tempFile. And you need to await that promise before proceeding ahead.

However it would be great if you also add some error handing code with the streams as mentioned in Error handling with node.js streams

like image 63
akshita007 Avatar answered Sep 18 '22 03:09

akshita007