Node js, piping pdfkit to a memory stream

Question

I am using pdfkit (https://github.com/devongovett/pdfkit) on my node server, typically creating pdf files, and then uploading them to s3. The problem is that pdfkit examples pipe the pdf doc into a node write stream, which writes the file to the disk, I followed the example and worked correctly, however my requirement now is to pipe the pdf doc to a memory stream rather than save it on the disk (I am uploading to s3 anyway). I've followed some node memory streams procedures but none of them seem to work with pdf pipe with me, I could just write strings to memory streams. So my question is: How to pipe the pdf kit output to a memory stream (or something alike) and then read it as an object to upload to s3?

var fsStream = fs.createWriteStream(outputPath + fileName); 
doc.pipe(fsStream);

Thanks in advance.

Answer 1

An updated answer for 2020. There is no need to introduce a new memory stream because "PDFDocument instances are readable Node streams".

You can use the get-stream package to make it easy to wait for the document to finish before passing the result back to your caller. https://www.npmjs.com/package/get-stream

const PDFDocument = require('pdfkit')
const getStream = require('get-stream')

const pdf = () => {
  const doc = new PDFDocument()
  doc.text('Hello, World!')
  doc.end()
  return await getStream.buffer(doc)
}


// Caller could do this:
const pdfBuffer = await pdf()
const pdfBase64string = pdfBuffer.toString('base64')

You don't have to return a buffer if your needs are different. The get-stream readme offers other examples.

Answer 2

There's no need to use an intermediate memory stream¹ – just pipe the pdfkit output stream directly into a HTTP upload stream.

In my experience, the AWS SDK is garbage when it comes to working with streams, so I usually use request.

var upload = request({
    method: 'PUT',
    url: 'https://bucket.s3.amazonaws.com/doc.pdf',
    aws: { bucket: 'bucket', key: ..., secret: ... }
});

doc.pipe(upload);

^{1 - in fact, it is usually undesirable to use a memory stream because that means buffering the entire thing in RAM, which is exactly what streams are supposed to avoid!}