Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Send a chunked HTTP response from a Go server

Tags:

I am creating a test Go HTTP server, and I am sending a response header of Transfer-Encoding: chunked so I can continually send new data as I retrieve it. This server should write a chunk to this server every one second. The client should be able to receive them on demand.

Unfortunately, the client(curl in this case), receives all of the chunks at the end of the duration, 5 seconds, rather than receiving one chunk every one second. Also, Go seems to send the Content-Length for me. I want to send the Content-Length at the end, and I want the the header's value to be 0.

Here is the server code:

package main  import (     "fmt"     "io"     "log"     "net/http"     "time" )  func main() {     http.HandleFunc("/test", HandlePost);     log.Fatal(http.ListenAndServe(":8080", nil)) }  func HandlePost(w http.ResponseWriter, r *http.Request) {     w.Header().Set("Connection", "Keep-Alive")     w.Header().Set("Transfer-Encoding", "chunked")     w.Header().Set("X-Content-Type-Options", "nosniff")      ticker := time.NewTicker(time.Second)     go func() {         for t := range ticker.C {             io.WriteString(w, "Chunk")             fmt.Println("Tick at", t)         }     }()     time.Sleep(time.Second * 5)     ticker.Stop()     fmt.Println("Finished: should return Content-Length: 0 here")     w.Header().Set("Content-Length", "0") } 
like image 286
kimura Avatar asked Nov 06 '14 00:11

kimura


People also ask

How do I send a chunked request?

Use the WEB SEND command to send the first chunk of the message. Specify CHUNKING(CHUNKYES) to tell CICS that it is a chunk of a message. Use the FROM option to specify the first chunk of data from the body of the message. Use the FROMLENGTH option to specify the length of the chunk.

What is http chunking in f5?

Chunking is a technique that HTTP servers use to improve responsiveness. Chunking can help you avoid situations where the server needs to obtain dynamic content from an external source and delays sending the response to the client until receiving all of the content so the server can calculate a Content-Length header.

How do I enable chunked transfer encoding?

To enable chunked transfer encoding, set the value for AspEnableChunkedEncoding to True in the metabase for the site, the server, or the virtual directory that you want to enable chunked transfer encoding for. By default, the value is True, and it is set at the Web service level.

What is http chunk size?

The chunk size is always given as a hexadecimal number. If that number is not directly followed by a CRLF, but a ; instead, you know that there is an extension. This extension is identified by its name ( chunk-ext-name ). If you never heard of that particular name, you MUST ignore it.


1 Answers

The trick appears to be that you simply need to call Flusher.Flush() after each chunk is written. Note also that the "Transfer-Encoding" header will be handled by the writer implicitly, so no need to set it.

func main() {   http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {     flusher, ok := w.(http.Flusher)     if !ok {       panic("expected http.ResponseWriter to be an http.Flusher")     }     w.Header().Set("X-Content-Type-Options", "nosniff")     for i := 1; i <= 10; i++ {       fmt.Fprintf(w, "Chunk #%d\n", i)       flusher.Flush() // Trigger "chunked" encoding and send a chunk...       time.Sleep(500 * time.Millisecond)     }   })    log.Print("Listening on localhost:8080")   log.Fatal(http.ListenAndServe(":8080", nil)) } 

You can verify by using telnet:

$ telnet localhost 8080 Trying ::1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1  HTTP/1.1 200 OK Date: Tue, 02 Jun 2015 18:16:38 GMT Content-Type: text/plain; charset=utf-8 Transfer-Encoding: chunked  9 Chunk #1  9 Chunk #2  ... 

You might need to do some research to verify that http.ResponseWriters support concurrent access for use by multiple goroutines.

Also, see this question for more information about the "X-Content-Type-Options" header.

like image 112
maerics Avatar answered Sep 17 '22 16:09

maerics