Safari rejects redirected CORS request due to browser-set headers

Question

Summary

Safari rejects some CORS requests which involve redirects, claiming that some header is not allowed. But that header is never requested by the script, but added by the browser, so I think it should not matter.

• Is Safari's behavior a bug,
• is there a problem with the specs, or
• is there a reason for things to be like this?

Error indicates some header is not allowed in CORS

I'm observing situations where Safari appears to interpret the CORS protocol more strictly than other browsers. It rejects certain requests with the error message

[Error] Failed to load resource: Request header field … is not allowed by Access-Control-Allow-Headers.

I've seen this for the header field Accept-Encoding, and after adding that field to Access-Control-Allow-Headers the same error was repeated with DNT. Other fields included in the Access-Control-Request-Headers list of the preflight request are Cache-Control, Origin and Accept-Language.

As far as I can tell, this only affects requests which get redirected. In cases where I managed to immediately name the final location, the request succeeded.

WHATWG XHR: Request should have no author-set headers

I created the XMLHttpRequest in my own code, without any library or framework interfering with it. I'm absolutely certain I never called the setRequestHeader() method.

Following the WHATWG XHR spec I would therefore assume that my author request headers list should be empty. The Request section states:

The author request headers is an initially empty header list.

I can read in the documentation that a non-empty argument to send() might trigger Content-Type as an author request header, but I'm experiencing problems even with such a no-argument call.

W3C CORS: Only author-set headers should matter

The W3C CORS recommendation describes the Access-Control-Request-Headers field as being composed of author request headers:

If author request headers is not empty include an Access-Control-Request-Headers header with as header field value a comma-separated list of the header field names from author request headers in lexicographical order, each converted to ASCII lowercase (even when one or more are a simple header).

And the likely reason for an error that I can see, according to that spec, would be this:

If the field name of each header in author request headers is not an ASCII case-insensitive match for one of the header field names in headers and the header is not a simple header, apply the cache and network error steps.

WHATWG Fetch takes the list from XHR

Perhaps the W3C CORS recommendation is the wrong place to look. Perhaps it has been superseded by the WHATWG Fetch living standard? That writes in its section about CORS-preflight fetch

3. Let headers be the names of request's header list's headers, excluding CORS-safelisted request-headers and duplicates, sorted lexicographically, and byte-lowercased.

4. Let value be the items in headers separated from each other by 0x2C.

5. Set Access-Control-Request-Headers to value in preflight's header list.

So what is the header list here? The connection between the living standards for XHR and Fetch appears to be the send() method specification.

6. Let req be a new request, initialized as follows:

   • method: request method
   • url: request URL
   • header list: author request headers
   • unsafe-request flag: Set.
   • […]

So it binds the author request headers (which according to my considerations above should be empty) to the header list which is used to build the list of requested headers. At least initially.

Headers might get added

Of course, the Fetch specification is long, and mentions the term “header list” in many places. It might well be that there is some clause somewhere in there which mandates adding certain header fields. But I see no portion where Accept-Encoding or DNT are explicitely mentioned as intended additions. They are listed as forbidden headers over which an author should have no control.

However, the following bit of text from the HTTP-network-or-cache fetch section mentions them in a note:

13. Modify httpRequest's header list per HTTP.

    Note: It would be great if we could make this more normative somehow. At this point headers such as Accept-Encoding, Connection, DNT, and Host, are to be appended if necessary.

    Accept, Accept-Charset, and Accept-Language must not be included at this point.

    Note: Accept and Accept-Language are already included (unless fetch() is used, which does not include the latter by default), and Accept-Charset is a waste of bytes. See HTTP header layer division for more details.

I find it very hard to judge the scope of this addition. Is this “adding headers per HTTP” the cause for extra headers being considered for CORS, in addition to those specified by the auther? But why only in case of redirects? The HTTP-redirect fetch section does not mention adding any more headers to the list of request headers.

Observed behavior would be bad

If the behavior I observe were according to the spec, then my main concern would be that the it would essentially prevent the addition of new and useful HTTP headers. You'd never know which sites might suddenly get broken as browsers start to not only add them but also include them in CORS checks. On the other hand, with only author-created scripts being considered, it would be sufficient to match request creation library to server implementation, with no worries about what browsers do. Sounds like a much saner setup to me.

Question

So I repeat my main question:

• Is Safari's behavior a bug,
• is there a problem with the specs, or
• is there a reason for things to be like this?

Workaround

For one server I just solved this by configuring my Apache to return all headers, but that's more of a workaround. I'd like to really understand what Safari is doing here, and why.

Cross links

There are several questions on SO indicating that Safari has problems with the combination of CORS and redirects:

• Safari Ajax cors request not following redirect (2013)
  Accepted answer states that “Apple claims that this is the way the HTML spec is written” without citing a source. Sounds like Webkit bug 112471 comment 2 except that answer predates the comment.

• HTML5 CORS request fails in safari after redirect (2014)
  Accepted self-answer proposes a server-assisted workaround.

• Safari fails CORS request after 302 redirect (2015)
  Unaccepted answer mentions webkit bug 98838 (unconfirmed) which in turn suggests bug 112471 (recently fixed). That one at some point also mentions bug 63460 titled “CORS should only deal with request headers set by script authors”, but doesn't seem to reach a definite consensus on this. Towards the end the DNT flag is explicitely named.

• Safari turns Simple Cors Request into Preflight after 302 redirect (2015)
  Unanswered.

The list of similar questions on the right might provide further information.

Answer 1

It might help if you split up your questions next time.

• WHATWG Fetch indeed obsoletes CORS (it defines CORS inline).
• CORS only pays attention to headers set before the network level (basically only those set at the API level). Some of these could be set by the browser, e.g., Accept, but I can't think of one that would affect CORS.

So yeah, I think Safari has a bug, but they have been making changes in this area very recently, so WebKit nightlies or Safari Technology Preview might do better.