Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

net/http server does not send 100 Continue unless client sends Transfer-Encoding: chunked or nonzero Content-Length

Tags:

http

go

I'm writing a HTTP server in Go that will receive requests from clients with a Expect: 100-continue header. However, it appears that the net/http server doesn't send a HTTP/1.1 100 Continue unless the client also sends a Transfer-Encoding: Chunked header, which some clients (for example, ffmpeg with an icecast:// destination) do not.

Here's a minimal server, that writes into a bytes.Buffer (I've reproduced the same behaviour with a more complicated server that, for example, uses io.Copy() to write into a file):

func main() {
    http.HandleFunc("/", func(writer http.ResponseWriter, r *http.Request) {
        log.Printf("Expect header: %v\n", r.Header.Get("Expect"))
        log.Printf("Transfer-Encoding header: %v\n", r.Header.Get("Transfer-Encoding"))

        buf := new(bytes.Buffer)
        defer func() {
            log.Printf("Buffer size: %d\n", buf.Len())
        }()
        defer r.Body.Close()

        log.Println("Writing.")
        io.Copy(buf, r.Body)
    })

    log.Fatal(http.ListenAndServe(":3948", nil))
}

And here's a transcript of two HTTP conversations (via telnet), where the server sends a 100 in one but not in the other:

PUT /telnetlol HTTP/1.1
Host: localhost
Expect: 100-continue

HTTP/1.1 200 OK
Date: Thu, 18 Mar 2021 10:59:09 GMT
Content-Length: 0
PUT /telnetlol HTTP/1.1
Host: localhost
Expect: 100-continue
Transfer-Encoding: chunked

HTTP/1.1 100 Continue

test
HTTP/1.1 200 OK
Date: Thu, 18 Mar 2021 10:59:35 GMT
Content-Length: 0
Connection: close

Is this a bug in Go, or am I misunderstanding the HTTP spec? The spec reads:

Upon receiving a request which includes an Expect request-header field with the "100-continue" expectation, an origin server MUST respond with 100 (Continue) status and continue to read from the input stream, or respond with a final status code. The origin server MUST NOT wait for the request body before sending the 100 (Continue) response.

Edit: Sending a non-zero Content-Length header in the initial request also makes the server reply with a 100 Continue. (Although, if I understand the spec correctly, it should still reply with a Continue irregardless.)

like image 932
Marks Polakovs Avatar asked Mar 18 '21 11:03

Marks Polakovs


People also ask

How do I enable chunked transfer encoding?

To enable chunked transfer encoding, you need to set the value of AspEnableChunkedEncoding to "True" in the metabase of the site, server, or virtual directory for which chunked transfer encoding is enabled. By default this value is set to "True", you can try to change the value to "False" to disable it.

What is a chunked HTTP response?

Chunking is a technique that HTTP servers use to improve responsiveness. Chunking can help you avoid situations where the server needs to obtain dynamic content from an external source and delays sending the response to the client until receiving all of the content so the server can calculate a Content-Length header.

What http request header is used to identify the acceptable Content types that can be returned?

The Accept request-header field can be used to specify certain media types which are acceptable for the response.

What is chunked header?

Chunked encoding allows the sender to send additional header fields after the message body. This is important in cases where values of a field cannot be known until the content has been produced, such as when the content of the message must be digitally signed.


1 Answers

The net/http server correctly handles the request:

PUT /telnetlol HTTP/1.1
Host: localhost
Expect: 100-continue

with this response:

HTTP/1.1 200 OK
Date: Thu, 18 Mar 2021 10:59:09 GMT
Content-Length: 0

The request does not have a message body per RFC 7230 3.3:

The presence of a message body in a request is signaled by a Content-Length or Transfer-Encoding header field.

A server may omit sending the a 100 response when there is no message body per RFC 7231 5.1.1:

A server MAY omit sending a 100 (Continue) response if it has already received some or all of the message body for the corresponding request, or if the framing indicates that there is no message body.

In addition, the client request is bad per RFC 7231 5.1.1:

A client MUST NOT generate a 100-continue expectation in a request that does not include a message body.

like image 91
Bayta Darell Avatar answered Oct 13 '22 19:10

Bayta Darell