Using an io.WriteSeeker without a File in Go

Question

I am using a third party library to generate PDFs. In order to write the PDF at the end (after all of content has been added using the lib's API), the pdfWriter type has a Write function that expects an io.WriteSeeker.

This is OK if I want to work with files, but I need to work in-memory. Trouble is, I can't find any way to do this - the only native type I found that implements io.WriteSeeker is File.

This is the part that works by using File for the io.Writer in the Write function of the pdfWriter:

fWrite, err := os.Create(outputPath)
if err != nil {
    return err
}

defer fWrite.Close()

err = pdfWriter.Write(fWrite)

Is there way to do this without an actual File? Like getting a []byte or something?

Answer 1

Unfortunately there is no ready solution for an in-memory io.WriteSeeker implementation in the standard lib.

But as always, you can always implement your own. It's not that hard.

An io.WriteSeeker is an io.Writer and an io.Seeker, so basically you only need to implement 2 methods:

Write(p []byte) (n int, err error)
Seek(offset int64, whence int) (int64, error)

Read the general contract of these methods in their documentation how they should behave.

Here's a simple implementation which uses an in-memory byte slice ([]byte). It's not optimized for speed, this is just a "demo" implementation.

type mywriter struct {
    buf []byte
    pos int
}

func (m *mywriter) Write(p []byte) (n int, err error) {
    minCap := m.pos + len(p)
    if minCap > cap(m.buf) { // Make sure buf has enough capacity:
        buf2 := make([]byte, len(m.buf), minCap+len(p)) // add some extra
        copy(buf2, m.buf)
        m.buf = buf2
    }
    if minCap > len(m.buf) {
        m.buf = m.buf[:minCap]
    }
    copy(m.buf[m.pos:], p)
    m.pos += len(p)
    return len(p), nil
}

func (m *mywriter) Seek(offset int64, whence int) (int64, error) {
    newPos, offs := 0, int(offset)
    switch whence {
    case io.SeekStart:
        newPos = offs
    case io.SeekCurrent:
        newPos = m.pos + offs
    case io.SeekEnd:
        newPos = len(m.buf) + offs
    }
    if newPos < 0 {
        return 0, errors.New("negative result pos")
    }
    m.pos = newPos
    return int64(newPos), nil
}

Yes, and that's it.

Testing it:

my := &mywriter{}
var ws io.WriteSeeker = my

ws.Write([]byte("hello"))
fmt.Println(string(my.buf))

ws.Write([]byte(" world"))
fmt.Println(string(my.buf))

ws.Seek(-2, io.SeekEnd)
ws.Write([]byte("k!"))
fmt.Println(string(my.buf))

ws.Seek(6, io.SeekStart)
ws.Write([]byte("gopher"))
fmt.Println(string(my.buf))

Output (try it on the Go Playground):

hello
hello world
hello work!
hello gopher

Things that can be improved:

• Create a mywriter value with an initial empty buf slice, but with a capacity that will most likely cover the size of the result PDF document. E.g. if you estimate the result PDFs are around 1 MB, create a buffer with capacity for 2 MB like this:
  my := &mywriter{buf: make([]byte, 0, 2<<20)}

• Inside mywriter.Write() when capacity needs to be increased (and existing content copied over), it may be profitable to use bigger increment, e.g. double the current capacity to a certain extent, which reserves space for future appends and minimizes the reallocations.