Time HTTP Response in Go

Question

In my Go program, I made some HTTP requests and I need to time the response time (and not request time).

Here is my current code (timing request time):

func Get() int {
    start := time.Now()
    result, err := http.Get("http://www.google.com")
    if err != nil {
        log.Fatal(err)
    }
    defer result.Body.Close()
    elapsed := time.Since(start).Seconds()
    log.Println(elapsed)

    return result.StatusCode
}

Actually, this code will show something about 5s request time, including DNS resolution and other things... If I execute the same test with a tool like Apache JMeter, the time is just about 100ms (which is the real response time of the server, without taking care about request time).

What I really want is to compute the real response time of the server. How can I compute this in Go ?

Answer 1

Not to take anything away from the perfectly valid accepted answer, one alternative to be aware of is to implement a custom RoundTripper that wraps the default http.Transport and net.Dialer. This can be helpful if you are instrumenting code that uses http.Client or if you need to support proxies, TLS, keep-alive or other HTTP capabilities but don't want/need to re-implement them all. You won't have quite as much control as you will with a fully customized client but it's worth having in your toolbox.

Example round tripper:

type customTransport struct {
    rtp       http.RoundTripper
    dialer    *net.Dialer
    connStart time.Time
    connEnd   time.Time
    reqStart  time.Time
    reqEnd    time.Time
}

func newTransport() *customTransport {
    tr := &customTransport{
        dialer: &net.Dialer{
            Timeout:   30 * time.Second,
            KeepAlive: 30 * time.Second,
        },
    }
    tr.rtp = &http.Transport{
        Proxy:               http.ProxyFromEnvironment,
        Dial:                tr.dial,
        TLSHandshakeTimeout: 10 * time.Second,
    }
    return tr
}

func (tr *customTransport) RoundTrip(r *http.Request) (*http.Response, error) {
    tr.reqStart = time.Now()
    resp, err := tr.rtp.RoundTrip(r)
    tr.reqEnd = time.Now()
    return resp, err
}

func (tr *customTransport) dial(network, addr string) (net.Conn, error) {
    tr.connStart = time.Now()
    cn, err := tr.dialer.Dial(network, addr)
    tr.connEnd = time.Now()
    return cn, err
}

func (tr *customTransport) ReqDuration() time.Duration {
    return tr.Duration() - tr.ConnDuration()
}

func (tr *customTransport) ConnDuration() time.Duration {
    return tr.connEnd.Sub(tr.connStart)
}

func (tr *customTransport) Duration() time.Duration {
    return tr.reqEnd.Sub(tr.reqStart)
}

I've dropped that into a simple example program here: https://github.com/skyec/go-instrumented-roundtripper/blob/master/main.go

Answer 2

Edit: The following answer pre-dates Go 1.7. Go 1.7 added HTTP tracing, so be sure to check out this new answer: Getting TTFB (time to first byte) value in golang

Original answer follows.

---

You can measure it by opening a TCP connection and "speaking" raw HTTP (wikipedia, rfc7230, rfc7231, rfc7232, rfc7233, rfc7234, rfc7235). You make a connection, send the request, and start the timer here. And wait for the response. By doing so you will exclude DNS resolving and the time required to make the connection.

Since you have no insight whether the server starts sending data back immediately or only when everything is ready, and you have no info about network delays, it won't be accurate rather just an estimation.

I would measure at 2 points: when the first byte can be read and when everything is read:

conn, err := net.Dial("tcp", "google.com:80")
if err != nil {
    panic(err)
}
defer conn.Close()
conn.Write([]byte("GET / HTTP/1.0\r\n\r\n"))

start := time.Now()
oneByte := make([]byte,1)
_, err = conn.Read(oneByte)
if err != nil {
    panic(err)
}
log.Println("First byte:", time.Since(start))

_, err = ioutil.ReadAll(conn)
if err != nil {
    panic(err)
}
log.Println("Everything:", time.Since(start))

Note:

It might be reasonable to measure at a 3rd point: when all the response headers are read. This is detected as reading full lines from the response (connection) and encountering an empty line. This empty line separates response headers from the response body.