Transparent (filter-like) gzip/gunzip in Go

Question

I'm trying, just for fun, to connect a gzip Writer directly to a gzip Reader, so I could write to the Writer and read from the Reader on the fly. I expected to read exactly what I wrote. I'm using gzip but I'd like to use this method also with crypto/aes, I suppose it should work very similar and it could be used with other reader/writers like jpeg, png...

This is my best option, that is not working, but I hope you can see what I mean: http://play.golang.org/p/7qdUi9wwG7

package main

import (
    "bytes"
    "compress/gzip"
    "fmt"
)

func main() {
    s := []byte("Hello world!")
    fmt.Printf("%s\n", s)

    var b bytes.Buffer

    gz := gzip.NewWriter(&b)
    ungz, err := gzip.NewReader(&b)
    fmt.Println("err: ", err)

    gz.Write(s)
    gz.Flush()
    uncomp := make([]byte, 100)
    n, err2 := ungz.Read(uncomp)
    fmt.Println("err2: ", err2)
    fmt.Println("n: ", n)
    uncomp = uncomp[:n]
    fmt.Printf("%s\n", uncomp)
}

It seems that gzip.NewReader(&b) is trying to read immediately and a EOF is returned.

Answer 1

You'll need to do two things to make it work

1. Use an io.Pipe to connect the reader and writer together - you can't read and write from the same buffer
2. Run the reading and writing in seperate goroutines. Because the first thing that gzip does is attempt to read the header you'll get a deadlock unless you have another go routine attemting to write it.

Here is what that looks like

Playground

func main() {
    s := []byte("Hello world!")
    fmt.Printf("%s\n", s)

    in, out := io.Pipe()

    gz := gzip.NewWriter(out)
    go func() {
        ungz, err := gzip.NewReader(in)
        fmt.Println("err: ", err)
        uncomp := make([]byte, 100)
        n, err2 := ungz.Read(uncomp)
        fmt.Println("err2: ", err2)
        fmt.Println("n: ", n)
        uncomp = uncomp[:n]
        fmt.Printf("%s\n", uncomp)
    }()
    gz.Write(s)
    gz.Flush()    
}