HTML5 iPhone dynamic caching of images

Question

We are building web applications for the iPhone that work offline. But we are having difficulties caching dynamic images. Please read on and I'll show by example exactly what I mean and what we have done so far.

So for example let's say we are building a simple list application with just 1 page. The application's only purpose is to list 5 items, each item containing some text and 1 image.

The application has a simple logo and some separate JavaScript and CSS code. These static resources are cached using the cache manifest file.

There are 2 scenario's:

Scenario 1: I'm online and I open up the web application

When I load the list application in Safari it will fetch 5 new random items out of a database containing 1000's of items. These are all served by simple backend through a AJAX call (JSON format).

The whole JSON object containing the 5 items is immediately stored in HTML5 local storage and cached for offline usage.

Structure of the JSON object is a bit like this:

{
    "0" : {
        id: "3",
        text: "Some text about item #3",
        image_url: "http://www.domain.com/image22341.png"
    },
    "1" : {
        id: "23",
        text: "Some text about item #23",
        image_url: "http://www.domain.com/image442321.png"
    },
    "2" : {
        id: "4",
        text: "Some text about item #4",
        image_url: "http://www.domain.com/image2321.png"
    },
    "3" : {
        id: "432",
        text: "Some text about item #432",
        image_url: "http://www.domain.com/image2441.png"
    },
    "4" : {
        id: "43",
        text: "Some text about item #43",
        image_url: "http://www.domain.com/image221.png"
    }
}

As you can see, very simple (may be some errors in that JSON), the whole JSON object is stored in local storage.

Now the 5 items are rendered using JavaScript injected HTML (styled using CSS), nothing fancy. Span tags containing the text and image tags pointing to the image resource are created etc.

In online mode, it all works great.

Scenario 2: I'm OFFLINE and I open up the web application

The page loads (the logo is displayed because it was cached as a static resource using the cache manifest), some JavaScript detects that we are indeed offline and the application as a result does not attempt to contact the backend. Instead it reads the previously stored JSON object from local storage and commences rendering the 5 items. All as anticipated.

Text is displayed fine, but this time, the images are not displayed, reason for that is simple, the image tags are pointing to image resources which are not available. So it displays that little image not available icon.

---

Now my question is, is there any way to cache those image resources somehow? So that next time we need them, they are fetched from cache.

This is what I've tried:

• Base64 encode the images and supply them through JSON. This works BUT, it dramatically increases both fetching and rendering time (we are talking about 30 seconds increase, very slow)
• Some cache manifest hacking/trial and error.. couldn't find anything that works (ideally need a policy that does 'cache all resources on domain as they are requested', but to my knowledge this does not exist)

I have literally spent hours on this and can't find a solution... does anyone have a clue? I know this is possible because if you look at the Google Mail HTML5 application for the iPhone, they can somehow cache the attachments and you can retrieve them even while offline.

The one thing we haven't tried is using the SQLite databases that are supported by Safari... maybe I could store the images as a BLOB (still means fetching it from the feed and thus slow?) and then somehow magically converting that into an image on the screen.... but I have no idea on how to do this.

Any help is appreciated, thanks.

Answer 1

Here is a code that downloads an image from AJAX and convert it to a base64 string. With this string you can save it to localStorage and assign it to the src property of an img in offline.

function _arrayBufferToBase64( buffer ) {
    var binary = '';
    var bytes = new Uint8Array( buffer );
    var len = bytes.byteLength;
    for (var i = 0; i < len; i++) {
        binary += String.fromCharCode( bytes[ i ] );
    }
    return window.btoa( binary );
}


var xhr = new XMLHttpRequest();
xhr.open('GET', 'an_image.png', true);
xhr.responseType = 'arraybuffer';

xhr.onload = function(e) {
        if (this.status == 200) {
          var string_with_bas64_image = _arrayBufferToBase64(this.response);

          // You can save this string to local storag or set it to an img elemment this way

          document.getElementById("img").src = "data:image/png;base64," + string_with_bas64_image;
        }
};

xhr.send();

You can test it here:http://jsbin.com/ivifiy/1/edit

With this technic you can write a localStorage cache.

The _arrayBufferToBase64 is extracted from here: ArrayBuffer to base64 encoded string

Answer 2

I would advise you to take a look to see if you can use canvases to hold your images, as they have certain properties to get/insert image pixel data, such as a CSV of pixel values (0-255).

Source: https://developer.apple.com/library/content/documentation/AudioVideo/Conceptual/HTML-canvas-guide/PixelManipulation/PixelManipulation.html

I'm not sure if you can dynamically use image sources into a canvas but if you can you should be able to transfer the CSVV data from the image to a DB and visa versa.

Quote

Safari 4.0 and later support direct manipulation of the pixels of a canvas. You can obtain the raw pixel data of a canvas with the getImageData() function and create a new buffer for manipulated pixels with the createImageData() function.

(source on archive.org)

---

Update:

I found this links you may also be interested in.

http://wecreategames.com/blog/?p=210

http://wecreategames.com/blog/?p=219 //Also note the idea with canvas serialization :)