How to pipe an HTTP response to a file in Go?

Question

How do I convert the below code to use streams/pipes so that I don't need to read the full content into memory? Something like: http.Get("http://example.com/").Pipe("./data.txt")

package main
import ("net/http";"io/ioutil")

func main() {
        resp, err := http.Get("http://example.com/")
        check(err)
        defer resp.Body.Close()
        body, err := ioutil.ReadAll(resp.Body)
        check(err)
        err = ioutil.WriteFile("./data.txt", body, 0666)
        check(err)
}
func check(e error) {
        if e != nil {
                panic(e)
        }
}

Answer 1

How about io.Copy()? Its documentation can be found at: http://golang.org/pkg/io/#Copy

It's pretty simple, though. Give it an io.Reader and an io.Writer and it copies the data over, one small chunk at a time (e.g. not all in memory at once).

So you might try writing something like:

func main() {
  resp, err := http.Get("...")
  check(err)
  defer resp.Body.Close()
  out, err := os.Create("filename.ext")
  if err != nil {
    // panic?
  }
  defer out.Close()
  io.Copy(out, resp.Body)
}

I haven't tested the above; I just hacked it together quickly from your above example, but it should be close if not on the money.

Answer 2

Another option is File.ReadFrom:

package main

import (
   "net/http"
   "os"
)

func main() {
   r, e := http.Get("http://speedtest.lax.hivelocity.net")
   if e != nil {
      panic(e)
   }
   defer r.Body.Close()
   f, e := os.Create("index.html")
   if e != nil {
      panic(e)
   }
   defer f.Close()
   f.ReadFrom(r.Body)
}