AJAX response gives a corrupted compressed (.tgz) file

Question

We are implementing a client-side web application that communicates with the server exclusively via XMLHttpRequests (and AJAX engine).

The XHR responses usually are plain text with some XML on it but in this case, the server is sending compressed data in .tgz file type. We know for sure that the data that the server is sending is correct because if we use an HTTP command-line client such as curl, the file sent as response is valid and contains the expected data.

However, when making an AJAX call and "blobing" the response in a downloadable file, the file we obtain is different in size (higher) than the correct one and it is not recognized by the decompresser. It Gives the following error:

gzip: stdin: not in gzip format
/bin/gtar: Child returned status 1
/bin/gtar: Error is not recoverable: exiting now

The code I'm using is the following:

*$.AJAX*.done(function(data){
    window.URL = window.webkitURL || window.URL;
    var contentType = 'application/x-compressed-tar';
    var file = new Blob([data], {type: contentType});
    var a = document.createElement('a'),
    ev = document.createEvent("MouseEvents");
    a.download = "browser_download2.tgz";
    a.href = window.URL.createObjectURL(file);
    ev.initMouseEvent("click", true, false, self, 0, 0, 0, 0, 0,
            false, false, false, false, 0, null);
    a.dispatchEvent(ev);
});

I avoided the parameters used to make the AJAX call, but let's assume that this is not the problem as I correctly receive an answer. I used this contentType because is the same one displayed by the obtained by curl but I tried different ones. The code may look a little bit weird so I'll desglosse it for you: I'm basically creating a link and I'm attaching to it the download link and the name of the file (it's a dirty way to be able to name the file). Finally I'm virtually clicking the link.

I compared the correct tgz file and the one obtained via browser with a hex viewer and I observed the repetition of patterns in the corrupted one (EF, BF and BD, all along the file) that is not present in the correct one.

Therefore I think about some possible causes:

(a) The browser is adding extra characters or maybe the response header is still in the downloaded file.

(b) The file has been partially decompressed because when I inspect the request Header I can state "Accept-Encoding: gzip, deflate"; although I don't know if the browser (Firefox in my case) automatically decompresses data.

(c) The code that I'm using to blob the data is not correct; although it acomplished well the aim with a plain/text file in another occasion.

Edit

I also provide you the links to the hex inspection:

(a) Corrupted file: http://en.webhex.net/view/278aac05820c34dfbdd2217c03970dd9/0 (b) (Presumably) correct file: http://en.webhex.net/view/4a01894b814c17d2ec71ba49ac48e683

Answer 1

I don't know if this thread will be helpful for somebody, but just in case I figured out the cause and a possible solution for my problem.

The cause

Default Javascript variables store information in Unicode/ASCII format; they are not prepared for storing binary data correctly and this is why one can easily see wrong characters interpreted (this also explains why repetitions of EF, BF, etc. were observed in the Hex Viewer, which stand for wrong characters of ASCII/Unicode).

The solution

The last browser versions implement the so called typed arrays. They are javascript arrays that can store data in different formats (also binary). Then, if one specifies that the XMLHttpRequest response is in binary format, data will be correctly stored and, when blobed into a file, the file will not be corrupted. Check out the code I used:

var xhr = new XMLHttpRequest();
xhr.open('POST', url, true);
xhr.responseType = 'arraybuffer';

Notice that the key point is to define the responseType as "arraybuffer". It may be also interesting noticing that I decided not to use Jquery for the AJAX anymore. It poorly implements this feature and all attempts I did to parse Jquery were in vain (overrideMimeType described somewhere else didn't work in my case). Instead, old plain XMLHttRquest worked pretty nicely.