Transferring large arrays to the client

Question

I need to transfer a large array of 16 bit integers to a client using an AJAX request to a web service. This array is essentially grey-scale pixel data for an image with some additional meta data.

The original image is in a proprietary format that is not supported by browsers. A typical image is 2000px x 4000px so the array could contain 8,000,000+ 16bit values.

Is it acceptable to make one AJAX request and return this entire array in one response or should I split into into smaller chunks and request each individually? If it is an accepted practice to transfer the whole array in one response, would I have to handle unreliable connections during the request (or does that come 'out of the box' in the browsers)?

We are writing both the client and the web service so we are entirely flexible on our approach.

Answer 1

Like others have mentioned, if you enable gzip compressed transfer in the server, the browser should decompress the image automatically. If the image is a scanned document with large areas of the same color this can actually compress the image quite a bit; if it's more like an aerial photograph then not so much.

Now that you have the image on the client side you have another problem: what to do with those 8 million 16-bit pixels.

Before we had typed arrays we had to use strings to represent arbitrary data. This is safer than it sounds because in JavaScript strings are sequences of 16-bit code units that may be interpreted as text encoded in UTF-16. JavaScript has no problem at all with strings with invalid encoding.

This means you can store your image as one long string, and access the individual pixels using charCodeAt. This should be reasonably fast for most manipulation purposes.

Here's an example of loading 16-bit binary data (actually a PNG icon) into a string and outputting the first 16 elements:

$.ajax({
    url:"http://fiddle.jshell.net/favicon.png",
    success: function(data) {
        // Now we have the data, let's show it:
        var out = "";
        for (var i = 0; i < 16; i++) {
            out += ","+data.charCodeAt(i).toString(16);
        }
        console.log(out.substring(1));
    },
    beforeSend: function ( xhr ) {
        xhr.overrideMimeType("text/plain; charset=utf-16be");
    },
    error: function(jqXHR, textStatus, errorThrown) {
        console.log("ERROR: "+textStatus+" "+errorThrown);
        console.log(jqXHR);
    },
})

(Live example at jsfiddle)