base64 encode io.Reader

Question

Is there a way to take an io.Reader that contains binary data, and read it out base64 encoded.

I see in encoding/base64 there is

func NewDecoder(enc *Encoding, r io.Reader) io.Reader

but that assumes the io.Reader data is base64 and returns an io.Reader to decode it to binary.

and

func NewEncoder(enc *Encoding, w io.Writer) io.WriteCloser

which returns a io.Writer to encode binary to base64 but I'm need to use the go-aws-sdk s3manage Uploader, which takes an io.Reader interface.

uploader := s3manager.NewUploaderWithClient(svc)
_, err := uploader.Upload(&s3manager.UploadInput{
   Bucket: aws.String(bucket),
  Key:    aws.String(key),
  Body:   input,
})

input needs implement the io.Reader interface

The data is large, so I don't want to read the input fully into memory before doing the encoding

Answer 1

The concept of a Pipe is used to change Readers into Writers and vice-versa.

Using an io.Pipe, you can copy the source io.Reader into the encoder, and pass the io.PipeReader as the body to be uploaded.

The error handling is a little unusual if you want to pass it through, but it's possible using the CloseWithError method. Make sure you also note the correct order to close the pipe and encoder.

source := strings.NewReader("Hello, World!")

pr, pw := io.Pipe()
encoder := base64.NewEncoder(base64.StdEncoding, pw)

go func() {
    _, err := io.Copy(encoder, source)
    encoder.Close()

    if err != nil {
        pw.CloseWithError(err)
    } else {
        pw.Close()
    }
}()

https://play.golang.org/p/qvc1f7kyTeP