Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How is an HTTP multipart "Content-length" header value calculated?

I've read conflicting and somewhat ambiguous replies to the question "How is a multipart HTTP request content length calculated?". Specifically I wonder:

  • What is the precise content range for which the "Content-length" header is calculated?
  • Are CRLF ("\r\n") octet sequences counted as one or two octets?

Can someone provide a clear example to answer these questions?

like image 852
Moshe Rubin Avatar asked Jul 14 '15 12:07

Moshe Rubin


People also ask

How is Content Length header calculated?

<? php print_r($_POST) ; ?>

How do I determine HTTP request header size?

go to the network tab and right click the first item and click copy as cURL (this is how you will get the header size. Then go to terminal and do your curl command curl ... -w '%{size_request} %{size_upload}' which will print out the request size at the end.

What is HTTP Content Length?

The content-length is the size of the compressed message body, in "octets" (i.e. in units of 8 bits, which happen to be "bytes" for all modern computers). The size of the actual message body can be something else, perhaps 150280 bytes.

Is Content Length required header?

The Content-Length header is mandatory for messages with entity bodies, unless the message is transported using chunked encoding. Content-Length is needed to detect premature message truncation when servers crash and to properly segment messages that share a persistent connection.


3 Answers

How you calculate Content-Length doesn't depend on the status code or media type of the payload; it's the number of bytes on the wire. So, compose your multipart response, count the bytes (and CRLF counts as two), and use that for Content-Length.

See: http://httpwg.org/specs/rfc7230.html#message.body.length

like image 127
Mark Nottingham Avatar answered Oct 14 '22 18:10

Mark Nottingham


The following live example should hopefully answer the questions.

Perform multipart request with Google's OAuth 2.0 Playground

Google's OAuth 2.0 Playground web page is an excellent way to perform a multipart HTTP request against the Google Drive cloud. You don't have to understand anything about Google Drive to do this -- I'll do all the work for you. We're only interested in the HTTP request and response. Using the Playground, however, will allow you to experiment with multipart and answer other questions, should the need arise.

Create a test file for uploading

I created a local text file called "test-multipart.txt", saved somewhere on my file system. The file is 34 bytes large and looks like this:

We're testing multipart uploading!

Open Google's OAuth 2.0 Playground

We first open Google's OAuth 2.0 Playground in a browser, using the URL https://developers.google.com/oauthplayground/:

Google OAuth 2.0 Playground opening screen

Fill in Step 1

Select the Drive API v2 and the "https://www.googleapis.com/auth/drive", and press "Authorize APIs":

Fields filled in for Step 1

Fill in Step 2

Click the "Exchange authorization code for tokens":

Fields filled in for Step 2

Fill in Step 3

Here we give all relevant multipart request information:

  • Set the HTTP Method to "POST"
  • There's no need to add any headers, Google's Playground will add everything needed (e.g., headers, boundary sequence, content length)
  • Request URI: "https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart"
  • Enter the request body: this is some meta-data JSON required by Google Drive to perform the multipart upload. I used the following:
{"title": "test-multipart.txt", "parents": [{"id":"0B09i2ZH5SsTHTjNtSS9QYUZqdTA"}], "properties": [{"kind": "drive#property", "key": "cloudwrapper", "value": "true"}]}
  • At the bottom of the "Request Body" screen, choose the test-multipart.txt file for uploading.
  • Press the "Send the request" button

enter image description here

The request and response

Google's OAuth 2.0 Playground miraculously inserts all required headers, computes the content length, generates a boundary sequence, inserts the boundary string wherever required, and shows us the server's response: enter image description here

Analysis

The multipart HTTP request succeeded with a 200 status code, so the request and response are good ones we can depend upon. Google's Playground inserted everything we needed to perform the multipart HTTP upload. You can see the "Content-length" is set to 352. Let's look at each line after the blank line following the headers:

--===============0688100289==\r\n
Content-type: application/json\r\n
\r\n
{"title": "test-multipart.txt", "parents": [{"id":"0B09i2ZH5SsTHTjNtSS9QYUZqdTA"}], "properties": [{"kind": "drive#property", "key": "cloudwrapper", "value": "true"}]}\r\n
--===============0688100289==\r\n
Content-type: text/plain\r\n
\r\n
We're testing multipart uploading!\r\n
--===============0688100289==--

There are nine (9) lines, and I have manually added "\r\n" at the end of each of the first eight (8) lines (for readability reasons). Here are the number of octets (characters) in each line:

  1. 29 + '\r\n'
  2. 30 + '\r\n'
  3. '\r\n'
  4. 167 + '\r\n'
  5. 29 + '\r\n'
  6. 24 + '\r\n'
  7. '\r\n'
  8. 34 + '\r\n' (although '\r\n' is not part of the text file, Google inserts it)
  9. 31

The sum of the octets is 344, and considering each '\r\n' as a single one-octet sequence gives us the coveted content length of 344 + 8 = 352.

Summary

To summarize the findings:

  1. The multipart request's "Content-length" is computed from the first byte of the boundary sequence following the header section's blank line, and continues until, and includes, the last hyphen of the final boundary sequence.
  2. The '\r\n' sequences should be counted as one (1) octet, not two, regardless of the operating system you're running on.
like image 35
Moshe Rubin Avatar answered Oct 14 '22 20:10

Moshe Rubin


If an http message has Content-Length header, then this header indicates exact number of bytes that follow after the HTTP headers. If anything decided to freely count \r\n as one byte then everything would fall apart: keep-alive http connections would stop working, as HTTP stack wouldn't be able to see where the next HTTP message starts and would try to parse random data as if it was an HTTP message.

like image 32
Pavel P Avatar answered Oct 14 '22 20:10

Pavel P