Detect end of HTTP request body

Question

I'm playing around with writing my own HTTP client and server and want to have the client include an optional body in the request. On the server side I want to read the entire body before sending the HTTP response. My question is on the server how do I know that I've read the entire body?

Even though in this case I control both the client and server, I'm looking for a "standard" approach. However, since Content-Length is optional I want a method that doesn't require it. If the client closes the connection, it is easy to read all available data, however the client needs to keep the connection open to wait for a response, so this method doesn't work.

All that I can think I'm left with is having knowledge of the format of the body and detecting a terminator (eg. </HTML>). Ideally I'm not wanting to require that knowledge.

Is there an approach I'm overlooking?

Answer 1

Assuming you want your client to work with other servers, and server to work with other clients, your server can't expect to be treated nicely.

There are two ways to tell when the body has ended. Neither of them require knowledge of the body's content type as you suggest (e.g., don't bother looking for </html> -- that goes far outside the HTTP protocol).

1. If the client sends a message with Transfer-Encoding: Chunked, you will need to parse the somewhat complicated chunked transfer encoding syntax. You don't really have much choice in the matter -- if the client is sending in this format, you have to receive it. When the client is using this approach, you can detect the end of the body by a chunk with a length of 0.
2. If the client instead sends a Content-Length, you must use that.

As you suggest, the third method for detecting the end -- when the connection closes -- only works for the response, not the request (as then there is no way to send a response).

Answer 2

If a request contains a message-body and a Content-Length is not given,  the server SHOULD respond with 400 (bad request) if it cannot determine the length of the message, or with 411 (length required) if it wishes  to insist on receiving a valid Content-Length. 

i.e. you are entitled to insist on either Transfer-Encoding: chunked or Content-Length, so you don't have to worry about determining the length in any other situation