Golang HTTP Concurrent Requests POST EOF

Question

I'm running a small client in Go. We're getting a number of false alarms at the moment, which seem to come down to client.Do() returning an EOF error when num>=1000.

This is the essence of my code:

func DoCreate(js string, cli *http.Client) {

    url2 := "http://xvz.myserver.com:9000/path/create"

    postBytesReader := bytes.NewReader([]byte(js))
    request, _ := http.NewRequest("POST", url2, postBytesReader)

    resp, err := cli.Do(request)
    if err != nil {
        fmt.Println(err) // Post http://xvz.myserver.com:9000/path/create: EOF 
        return
    }
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))

    defer resp.Body.Close()
}

func main() {
    client := &http.Client{
        CheckRedirect: nil,
    }

    for i := 0; i < num; i++ {
        var pq CreateReqInfo
        pq.Uid = strconv.Itoa(bid + i)
        petstr, _ := json.Marshal(pq)
        go DoCreate(string(petstr), client)
    }
}

Is problem about number of file handles or max connections?

Answer 1

It looks to me that you might be having the issue described in this answer. Basically, the Go http client will try to reuse connections unless you've specifically indicated that it shouldn't either in the Transport you've set your Client instance to use or on the request itself with:

request.Close = true

This becomes a problem when the server on the other end closes the connection without indicating so with a Connection: close header in a response. The net.http code assumes that the connection is still open and so the next request that tries to use the connection next, encounters the EOF from the connection being closed the other time.

You would need to check what is happening on the connection around the 1000th request. Is the receiving server set to limit the number of connections per client? Is your code running into a connection timeout? In either case, if it so happens the connection is closed by the server in a way that the Go Client code can't predict, then you'll run into the EOF.

Answer 2

• First, what for you instantiate new client each time client := &http.Client{... inside DoCreate() inside a loop? Client can be reusable even concurrently, so you can construct it more globally, say in main() seems to me.
• Then for me such error looks like produced by RoundTrip, so by connection, so possibly from server site. Can you test with mocking server?
• Finally if all that would not help, then yes, some systems have restriction on number of opened FileDescriptors which every net.Conn wants to have own. This restriction can be eliminated only on OS level.

Answer 3

The EOF is usually from a server not returning a complete header (including both CRLFs), or the connection is closed before the header is complete. It's more likely that your overloading your server with concurrent requests, though you should still make sure you have adequate resources locally for the number of concurrent requests you're making. If num is large enough, you're going to run out of something.

Though the error isn't really descriptive, it's not anything to worry about more so than any other request error. It's an error condition, and handle it like you would any others. If you want to know for certain, you'll likely have to have a packet capture of the connection that cause the EOF.