Golang HTTP server wait for data to send to client

Question

I am creating a streaming API similar to the Twitter firehose/streaming API.

As far as I can gather this is based on HTTP connections that are kept open and when the backend gets data it then writes to the chucked HTTP connection. It seems that any code I write closes the HTTP connection as soon as anything connects.

Is there a way to keep this open at all?

func startHTTP(pathPrefix string) {
    log.Println("Starting HTTPS Server")
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        // Wait here until a write happens to w
        // Or we timeout, we can reset this timeout after each write
    })

    log.Print("HTTPS listening on :5556")
    log.Fatal(http.ListenAndServeTLS(":5556", pathPrefix+".crt", pathPrefix+".key", nil))
}

Answer 1

When you want to send HTTP response to client not immediately but after some event, it's called long polling.

Here's simple example of long polling with request cancellation on client disconnect:

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"
)

func longOperation(ctx context.Context, ch chan<- string) {
    // Simulate long operation.
    // Change it to more than 10 seconds to get server timeout.
    select {
    case <-time.After(time.Second * 3):
        ch <- "Successful result."
    case <-ctx.Done():
        close(ch)
    }
}

func handler(w http.ResponseWriter, _ *http.Request) {
    notifier, ok := w.(http.CloseNotifier)
    if !ok {
        panic("Expected http.ResponseWriter to be an http.CloseNotifier")
    }

    ctx, cancel := context.WithCancel(context.Background())
    ch := make(chan string)
    go longOperation(ctx, ch)

    select {
    case result := <-ch:
        fmt.Fprint(w, result)
        cancel()
        return
    case <-time.After(time.Second * 10):
        fmt.Fprint(w, "Server is busy.")
    case <-notifier.CloseNotify():
        fmt.Println("Client has disconnected.")
    }
    cancel()
    <-ch
}

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe("localhost:8080", nil)
}

URLs:

1. anonymous struct and empty struct.
2. Send a chunked HTTP response from a Go server.
3. Go Concurrency Patterns: Context.

Gists:

1. Golang long polling example.
2. Golang long polling example with request cancellation.