XMLHttpRequest CORS to Google Cloud Storage only working in preflight request

Question

I implemented browser based resumable uploads into Google's Cloud Storage using an XMLHttpRequest send to a server-side created resumable upload url. This works completely fine when disabling web security, which I did during development.

But now in the real world, CORS keeps making trouble. I tried this with other browsers (without success), too, but sticked to chrome for further testing.

Note: A fake.host entry in /etc/HOSTS is used to trick chrome into avoiding localhost-restrictions. Nevertheless, same happens with the "real" domain of our online test server.

The request is started using a normal XMLHttpRequest call:

var xhr = this.newXMLHttpRequest();
xhr.open('PUT', url, true);

xhr.setRequestHeader('Content-Type', this.currentInputFile.type);
xhr.setRequestHeader('Content-Range', 'bytes ' + startByte + '-' + (this.currentInputFile.size - 1) + '/' + this.currentInputFile.size);

xhr.onload = function(e) {
   ...
};

...

if (startByte > 0) {
    xhr.send(this.currentInputFile.slice(startByte));
} else {
    xhr.send(this.currentInputFile);
}

The browser then successfully initiates a preflight request:

Remote Address:173.194.71.95:443
Request URL:https://www.googleapis.com/upload/storage/v1/b/my-bucket-name/o?uploadType=resumable&name=aa%20spacetestSMALL_512kb.mp4&upload_id=XXXXXXXXX
Request Method:OPTIONS
Status Code:200 OK

Request Headers:

:host:www.googleapis.com
:method:OPTIONS
:path:/upload/storage/v1/b/my-bucket-name/o?uploadType=resumable&name=aa%20spacetestSMALL_512kb.mp4&upload_id=XXXXXXXXX
:scheme:https
:version:HTTP/1.1
accept:*/*
accept-encoding:gzip,deflate
accept-language:en-US,en;q=0.8,de;q=0.6
access-control-request-headers:content-range, content-type
access-control-request-method:PUT
origin:https://fake.host
referer:https://fake.host/upload.xhtml
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 Safari/537.36
x-client-data:YYYYYY

Query String Parameters

uploadType:resumable
name:aa spacetestSMALL_512kb.mp4
upload_id:XXXXXXXXX

Response Headers

access-control-allow-credentials:true
access-control-allow-headers:content-range, content-type
access-control-allow-methods:PUT
access-control-allow-origin:https://fake.host
alternate-protocol:443:quic
content-length:0
content-type:text/html; charset=UTF-8
date:Fri, 05 Sep 2014 14:11:21 GMT
server:UploadServer ("Built on Aug 18 2014 11:58:36 (1408388316)")
status:200 OK
version:HTTP/1.1

... and starts the PUT-request until all data is transferred. But afterwards chrome silently logs an error without completing/ending the request:

XMLHttpRequest cannot load https://www.googleapis.com/upload/storage/v1/b/my-bucket-name…XXXXXXXX. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://fake.host' is therefore not allowed access.

This is what chrome logs about the PUT request:

Request URL:https://www.googleapis.com/upload/storage/v1/b/my-bucket-name/o?uploadType=resumable&name=aa%20spacetestSMALL_512kb.mp4&upload_id=XXXXXXXXX

Request Headers

Provisional headers are shown
Content-Range:bytes 0-3355302/3355303
Content-Type:video/mp4
Origin:https://fake.host
Referer:https://fake.host/upload.xhtml
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 Safari/537.36
X-DevTools-Emulate-Network-Conditions-Client-Id:YYYYYYY

Query String

uploadType:resumable
name:aa spacetestSMALL_512kb.mp4
upload_id:XXXXXXXXX

Notably, when adding the same url in http://client.cors-api.appspot.com/client and issuing any request, all but the OPTIONS request types fail, too. It seems like the cloud storage api only issues the correct response headers for OPTION requests, but not PUT/POST/GET/... requests.

So am I doing something impossible? Is there something broken? Is this a bug in the cloud storage api? I've spend hours googling and reading SO answers, without any luck so far.

For now, I could periodically check if the download transferred 100% of the data and just ignore the http request outcome, as the file is in fact completely uploaded to the storage bucket. But this is a rather ugly workaround which I really don't want to use if the real issue can be solved.

Answer 1

When requesting a resumable upload url, you MUST include the origin the browser will send when trying to use that upload url, or else the subsequent uploading will fail, just as is happening in the question (the OPTIONS call will look good, but the PUT will not).

It must exactly match the browser's origin (which you can get as location.origin in javascript).

This is the step "Initiating a resumable upload session" in this documentation: https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload

If you're requesting the resumable upload url on the server side, you'll probably need the client side (the browser) to pass you its origin (eg: location.origin).

fwiw I was using Google's Cloud Storage library for python for this step, and needed to add the origin like this:

myblob.create_resumable_upload_session(mycontenttype, origin=browserorigin)

Note that you definitely do not need to set up CORS for your bucket.

Answer 2

Since this question remains unanswered and still gets a fair number of view I'll try to post something definitive here.

The 'Access-Control-Allow-Origin' header returned in the response to any PUT requests to upload data is always set to the the origin given in the initial POST request used to initiate the upload, as per the current docs:

When using the resumable upload protocol, the Origin from the first (start upload) request is always used to decide the Access-Control-Allow-Origin header in the response, even if you use a different Origin for subsequent request. Therefore, you should either use the same origin for the first and subsequent requests, or if the first request has a different origin than subsequent requests, use the XML API with the CORS configuration set to *.

This means you must send an initial POST request before any PUT requests that send data, and any subsequent PUT requests must have the same 'origin' as the initial POST.

Regarding the CORS configuration set in GCS, this only applies to calls to the XML API, from the current docs:

Note: CORS configuration applies only to XML API requests. For JSON API requests, Cloud Storage always returns the Access-Control-Allow-Origin header with the origin of the request.