Golang http request results in EOF errors when making multiple requests successively

Question

I am trying to debug a very unusual error I am receiving for a simple REST library I wrote.

I am using the standard net/http package to make Get, Post, Put, Delete requests but my tests occasionally fail when I make multiple requests successively. My test looks like this:

func TestGetObject(t *testing.T) {     firebaseRoot := New(firebase_url)     body, err := firebaseRoot.Get("1")     if err != nil {         t.Errorf("Error: %s", err)     }     t.Logf("%q", body) }    func TestPushObject(t *testing.T) {     firebaseRoot := New(firebase_url)     msg := Message{"testing", "1..2..3"}     body, err := firebaseRoot.Push("/", msg)     if err != nil {         t.Errorf("Error: %s", err)     }     t.Logf("%q", body) } 

And I am making the request like this:

// Send HTTP Request, return data func (f *firebaseRoot) SendRequest(method string, path string, body io.Reader) ([]byte, error) { url := f.BuildURL(path)  // create a request req, err := http.NewRequest(method, url, body) if err != nil {     return nil, err }  // send JSON to firebase resp, err := http.DefaultClient.Do(req) if err != nil {     return nil, err }  if resp.StatusCode != http.StatusOK {     return nil, fmt.Errorf("Bad HTTP Response: %v", resp.Status) }  defer resp.Body.Close() b, err := ioutil.ReadAll(resp.Body) if err != nil {     return nil, err }  return b, nil }  

Sometimes it works, but most of the time I get 1 or 2 failures:

--- FAIL: TestGetObject (0.00 seconds) firebase_test.go:53: Error: Get https://go-firebase-test.firebaseio.com/1.json: EOF firebase_test.go:55: ""  --- FAIL: TestPushObject (0.00 seconds) firebase_test.go:63: Error: Post https://go-firebase-test.firebaseio.com/.json: EOF firebase_test.go:65: "" FAIL exit status 1 FAIL    github.com/chourobin/go.firebase    3.422s 

The failures happen when I make more than 1 request. If I comment out everything except for the PUT request, the tests consistently pass. Once I include a second test, such as GET, one or the other fails (sometimes both pass).

Answer 1

I experienced this reliably. You need to set Req.Close to true (the defer on resp.Body.Close() syntax used in the examples is not enough). Like this:

client := &http.Client{} req, err := http.NewRequest(method, url, httpBody)  // NOTE this !! req.Close = true  req.Header.Set("Content-Type", "application/json") req.SetBasicAuth("user", "pass") resp, err := client.Do(req) if err != nil {     // whatever } defer resp.Body.Close()  response, err = ioutil.ReadAll(resp.Body) if err != nil {     // Whatever } 

Answer 2

I agree with the assertion that you shouldn't be hitting outside servers in your unit tests, why not just use the built-in http.Server and serve up the content that you want to test. (There is actually the httptest package to help with this)

I recently ran into this same problem while trying to crawl sitemaps, and this is what I have found so far:

Go by default will send requests with the header Connection: Keep-Alive and persist connections for re-use. The problem that I ran into is that the server is responding with Connection: Keep-Alive in the response header and then immediately closing the connection.

As a little background as to how go implements connections in this case (you can look at the full code in net/http/transport.go). There are two goroutines, one responsible for writing and one responsible for reading (readLoop and writeLoop) In most circumstances readLoop will detect a close on the socket, and close down the connection. The problem here occurs when you initiate another request before the readLoop actually detects the close, and the EOF that it reads get interpreted as an error for that new request rather than a close that occurred prior to the request.

Given that this is the case the reason why sleeping in between requests works is that it gives readLoop time to detect the close on the connection before your new request and shut it down, so that your new request will initiate a new connection. (And the reason why it would intermittently fail is because there is some amount code running between your requests and depending of scheduling of goroutines, sometimes the EOF will be properly handled before your next request, sometimes not). And the req.Close = true, solution works because it prevents the connection from being re-used.

There is a ticket related to this situation: https://code.google.com/p/go/issues/detail?id=4677 (and a dupe ticket that I created that allowed me to reliably reproduce this: https://code.google.com/p/go/issues/detail?id=8122)