Specify timeout when tracing HTTP request in Go

Question

I know the usual method of specifying a timeout with HTTP requests by doing:

httpClient := http.Client{
    Timeout: time.Duration(5 * time.Second),
}

However, I can't seem to figure out how to do the same when tracing HTTP requests. Here is the piece of code I am working with:

func timeGet(url string) (httpTimingBreakDown, error) {
    req, _ := http.NewRequest("GET", url, nil)

    var start, connect, dns, tlsHandshake time.Time
    var timingData httpTimingBreakDown
    timingData.url = url

    trace := &httptrace.ClientTrace{
        TLSHandshakeStart:    func() { tlsHandshake = time.Now() },
        TLSHandshakeDone:     func(cs tls.ConnectionState, err error) { timingData.tls = time.Since(tlsHandshake) },
    }

    req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
    start = time.Now()

    http.DefaultTransport.(*http.Transport).ResponseHeaderTimeout = time.Second * 10 // hacky way, worked earlier but don't work anymore

    if _, err := http.DefaultTransport.RoundTrip(req); err != nil {
        fmt.Println(err)
        return timingData, err
    }

    timingData.total = time.Since(start)

    return timingData, nil
}

I am firing this function inside a goroutine. My sample data set is 100 urls. All goroutines fire, but eventually the program ends in 30+ secs as if the timeout is 30secs.

Earlier I made the same to work by using the hacky way of changing the default inside of it to 10 secs and anything that took too long, timed out and the program ended at 10.xxx secs but now its taking 30.xx secs.

What would be a proper way of specifying a timeout in this scenario?

Answer 1

I know the usual method of specifying a timeout with HTTP requests by doing:

httpClient := http.Client{
    Timeout: time.Duration(5 * time.Second),
}

Actually, the preferred method is to use a context.Context on the request. The method you've used is just a short-cut suitable for simple use cases.

req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
    return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()
req = req.WithContext(ctx)

And this method should work nicely for your situation as well.