Chunked Transfer and multipart http

Question

in HTTP request and response paradigm : Is it possible to use chunked transfer encoding with multipart HTTP messages? If no Why?

Answer 1

It looks like the entire request body is subject to chunking, and structuring (or providing alternative versions of) multipart data appears orthogonal to the chunking. I can see curl wrapping entire parts into chunks,

$ curl --trace-ascii - https://httpbin.org/post --form "field1=Bonjour." -H "Expect:" -H "User-Agent:" -H "Accept:" --http1.1
[...]
=> Send header, 149 bytes (0x95)
0000: POST /post HTTP/1.1
0015: Host: httpbin.org
0028: Content-Length: 149
003d: Content-Type: multipart/form-data; boundary=--------------------
007d: ----5cb7d87e27c5c1bf
0093:
== Info: TLSv1.2 (OUT), TLS header, Supplemental data (23):
=> Send SSL data, 5 bytes (0x5)
0000: .....
=> Send data, 149 bytes (0x95)
0000: --------------------------5cb7d87e27c5c1bf
002c: Content-Disposition: form-data; name="field1"
005b:
005d: Bonjour.
0067: --------------------------5cb7d87e27c5c1bf--
== Info: We are completely uploaded and fine
[...]

$ curl --trace-ascii - https://httpbin.org/post --form "field1=Bonjour." -H "Expect:" -H "User-Agent:" -H "Accept:" --http1.1 -H "Transfer-Encoding: chunked"
[...]
=> Send header, 156 bytes (0x9c)
0000: POST /post HTTP/1.1
0015: Host: httpbin.org
0028: Transfer-Encoding: chunked
0044: Content-Type: multipart/form-data; boundary=--------------------
0084: ----db8064ac2c1d04aa
009a:
== Info: TLSv1.2 (OUT), TLS header, Supplemental data (23):
=> Send SSL data, 5 bytes (0x5)
0000: .....
=> Send data, 155 bytes (0x9b)
0000: 95
0004: --------------------------db8064ac2c1d04aa
0030: Content-Disposition: form-data; name="field1"
005f:
0061: Bonjour.
006b: --------------------------db8064ac2c1d04aa--
0099:
== Info: Signaling end of chunked upload via terminating chunk.
[...]
=> Send data, 5 bytes (0x5)
0000: 0
0003:
[...]

It's possible to split the request body into arbitrary chunks.

$ cr=$'\r'; lf=$'\n'; delim="xyz:abc:foobar"; h="httpbin.org"; body="--${delim}${cr}${lf}Content-Disposition: form-data; name=\"field1\"${cr}${lf}${cr}${lf}Bonjour.${cr}${lf}--${delim}--${cr}${lf}"; cl=${#body}; { echo -ne "POST /post HTTP/1.1\r\nHost: ${h}\r\nContent-Type: multipart/form-data; boundary=${delim}\r\nContent-Length: ${cl}\r\n\r\n${body}"; sleep 2; } | tee /dev/stderr | openssl s_client -connect "${h}:443" -servername "${h}" -quiet -no_ign_eof
POST /post HTTP/1.1
Host: httpbin.org
Content-Type: multipart/form-data; boundary=xyz:abc:foobar
Content-Length: 97

--xyz:abc:foobar
Content-Disposition: form-data; name="field1"

Bonjour.
--xyz:abc:foobar--
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = httpbin.org
verify return:1
HTTP/1.1 200 OK
Date: Thu, 27 Oct 2022 03:57:21 GMT
Content-Type: application/json
Content-Length: 388
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "field1": "Bonjour."
  },
  "headers": {
    "Content-Length": "97",
    "Content-Type": "multipart/form-data; boundary=xyz:abc:foobar",
    "Host": "httpbin.org",
    "X-Amzn-Trace-Id": "Root=1-635a01a1-569f3ccb69a8ad4d07704cc7"
  },
  "json": null,
  "origin": "WWW.XXX.YYY.ZZZ",
  "url": "https://httpbin.org/post"
}
DONE

$ cr=$'\r'; lf=$'\n'; delim="xyz:abc:foobar"; h="httpbin.org"; body="--${delim}${cr}${lf}Conte
nt-Disposition: form-data; name=\"field1\"${cr}${lf}${cr}${lf}Bonjour.${cr}${lf}--${delim}--${cr}${lf}"; chunked=""; whi
le true; do b="${body:0:32}"; body="${body:32}"; chunked+="$(printf "%x" ${#b})${cr}${lf}${b}${cr}${lf}"; (( ${#b} )) ||
 break; done; cl=${#chunked}; { echo -ne "POST /post HTTP/1.1\r\nHost: ${h}\r\nContent-Type: multipart/form-data; bounda
ry=${delim}\r\nTransfer-Encoding: chunked\r\nContent-Length: ${cl}\r\n\r\n${chunked}"; sleep 2; } | tee /dev/stderr | op
enssl s_client -connect "${h}:443" -servername "${h}" -quiet -no_ign_eof
POST /post HTTP/1.1
Host: httpbin.org
Content-Type: multipart/form-data; boundary=xyz:abc:foobar
Transfer-Encoding: chunked
Content-Length: 125

20
--xyz:abc:foobar
Content-Dispos
20
ition: form-data; name="field1"
20


Bonjour.
--xyz:abc:foobar--
1


0

depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = httpbin.org
verify return:1
HTTP/1.1 200 OK
Date: Thu, 27 Oct 2022 04:07:47 GMT
Content-Type: application/json
Content-Length: 388
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "field1": "Bonjour."
  },
  "headers": {
    "Content-Length": "97",
    "Content-Type": "multipart/form-data; boundary=xyz:abc:foobar",
    "Host": "httpbin.org",
    "X-Amzn-Trace-Id": "Root=1-635a0412-6a407dba17a44c4b09c49a36"
  },
  "json": null,
  "origin": "76.71.106.87",
  "url": "https://httpbin.org/post"
}
DONE