Reusing http connections in Go

Question

I'm currently struggling to find a way to reuse connections when making HTTP posts in Go.

I've created a transport and client like so:

// Create a new transport and HTTP client tr := &http.Transport{} client := &http.Client{Transport: tr} 

I'm then passing this client pointer into a goroutine which is making multiple posts to the same endpoint like so:

r, err := client.Post(url, "application/json", post) 

Looking at netstat this appears to be resulting in a new connection for every post resulting in a large number of concurrent connections being open.

What is the correct way to reuse connections in this case?

Answer 1

Ensure that you read until the response is complete AND call Close().

e.g.

res, _ := client.Do(req) io.Copy(ioutil.Discard, res.Body) res.Body.Close() 

Again... To ensure http.Client connection reuse be sure to:

• Read until Response is complete (i.e. ioutil.ReadAll(resp.Body))
• Call Body.Close()

Answer 2

If anyone is still finding answers on how to do it, this is how I am doing it.

package main  import (   "bytes"   "io/ioutil"   "log"   "net/http"   "time" )  func httpClient() *http.Client {     client := &http.Client{         Transport: &http.Transport{             MaxIdleConnsPerHost: 20,         },         Timeout: 10 * time.Second,     }      return client }  func sendRequest(client *http.Client, method string) []byte {     endpoint := "https://httpbin.org/post"     req, err := http.NewRequest(method, endpoint, bytes.NewBuffer([]byte("Post this data")))     if err != nil {         log.Fatalf("Error Occured. %+v", err)     }      response, err := client.Do(req)     if err != nil {         log.Fatalf("Error sending request to API endpoint. %+v", err)     }      // Close the connection to reuse it     defer response.Body.Close()      body, err := ioutil.ReadAll(response.Body)     if err != nil {         log.Fatalf("Couldn't parse response body. %+v", err)     }      return body }  func main() {     c := httpClient()     response := sendRequest(c, http.MethodPost)     log.Println("Response Body:", string(response)) } 

Go Playground: https://play.golang.org/p/cYWdFu0r62e

In summary, I am creating a different method to create an HTTP client and assigning it to a variable, and then using it to make requests. Note the

defer response.Body.Close()  

This will close the connection after the request is complete at the end of the function execution and you can reuse the client as many times.

If you want to send a request in a loop call the function that sends the request in a loop.

If you want to change anything in the client transport configuration, like add proxy config, make a change in the client config.

Hope this will help someone.