Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

HTTP POST using XHR with Chunked Transfer Encoding

I have a REST API that accepts an Audio file via an HTTP Post. The API has support for Transfer-Encoding: chunked request header so that the file can be uploaded in pieces as it is being created from a recorder running on the client. This way the server can start processing the file as it arrives for improved performance. For example:

HTTP 1.1 POST .../v1/processAudio

Transfer-Encoding: chunked

[Chunk 1 256 Bytes] (server starts processing when arrives)

[Chunk 2 256 Bytes]

[Chunk 3 256 Bytes]

...

The audio files are typically short and are around 10K to 100K in size. I have C# and Java code that is working so I know that API works. However, I cannot seem to get the recording and upload working in a browser using javascript.

Here is my Test Code that does a POST to localhost with Transfer-Encoding:

<html>
<script type="text/javascript">
  function streamUpload() {
    var blob = new Blob(['GmnQPBU+nyRGER4JPAW4DjDQC19D']);
    var xhr = new XMLHttpRequest();
    // Add any event handlers here...
    xhr.open('POST', '/', true);
    xhr.setRequestHeader("Transfer-Encoding", "chunked");
    xhr.send(blob);
  }
</script>

<body>
  <div id='demo'>Test Chunked Upload using XHR</div>
  <button onclick="streamUpload()">Start Upload</button>
</body>

</html>

The problem is that i'm receiving the following Error in Chrome

Refused to set unsafe header "Transfer-Encoding"

streamUpload @ uploadTest.html:14 onclick @ uploadTest.html:24

After looking at XHR documentation i'm still confused because it does not talk about unsafe request headers. I'm wondering if its possible that XHR does not allow or implement Transfer-Encoding: chunked for HTTP POST?

I've looked at work arounds using multiple XHR.send() requests and WebSockets but both are undesirable because it will require significant changes to the server APIs which are already in place, simple, stable and working. The only issue is that we cannot seem to POST from a browser with psedo-streaming via Transfer-Encoding: chunked request header.

Any thoughts or advice would be very helpful.

like image 707
Mike Kennewick Avatar asked Jul 22 '15 16:07

Mike Kennewick


People also ask

What is the use of Transfer-Encoding chunked?

Chunked transfer encoding allows a server to maintain an HTTP persistent connection for dynamically generated content. In this case, the HTTP Content-Length header cannot be used to delimit the content and the next HTTP request/response, as the content size is not yet known.

How do I send a chunked request?

Use the WEB SEND command to send the first chunk of the message. Specify CHUNKING(CHUNKYES) to tell CICS that it is a chunk of a message. Use the FROM option to specify the first chunk of data from the body of the message. Use the FROMLENGTH option to specify the length of the chunk.

How do you stop Transfer-Encoding chunked?

Try adding "&headers=false" to your request. That should shorten it up and cause the response to be less likely to be chunked. Also, are you sending a HTTP/1.1 or HTTP/1.0 request? Try sending a HTTP/1.0 if your device cannot handle a HTTP/1.1 request.


1 Answers

As was mentioned in a comment, you're not allowed to set that header as it's controlled by the user agent.

For the full set of headers, see 4.6.2 The setRequestHeader() method from W3C XMLHttpRequest Level 1 and note that Transfer-Encoding is one of the headers that are controlled by the user agent to let it control those aspects of transport.

  • Accept-Charset
  • Accept-Encoding
  • Access-Control-Request-Headers
  • Access-Control-Request-Method
  • Connection
  • Content-Length
  • Cookie
  • Cookie2
  • Date
  • DNT
  • Expect
  • Host
  • Keep-Alive
  • Origin
  • Referer
  • TE
  • Trailer
  • Transfer-Encoding
  • Upgrade
  • User-Agent
  • Via

There is a similar list in the WhatWG Fetch API Living Standard. https://fetch.spec.whatwg.org/#terminology-headers

like image 106
Kevin Hakanson Avatar answered Sep 25 '22 12:09

Kevin Hakanson