loading bpg images: compatible, simple and efficient techniques

Question

Recently there has been some hubbub regarding Fabrice Bellard's BPG image format (http://bellard.org/bpg/), which (based on the demos provided on his site) provides better compression than jpeg, webp and some others. The image decoding is done in the browser with JS, which means it can be used immediately, without waiting for browser adoption. Overall this seems like a good idea and trading some CPU time for faster downloading is often a workable tradeoff.

The technique being used here to swap out images is to, on window.load, iterate over document.images, find any where the src attribute contains a URL ending with ".bpg" and replace that with a canvas.

This is however definitely not the only way to approach the problem, and I see some down sides to this technique, which include: a) canvases do not have exactly the same layout rules as images - e.g. setting the width attribute on it means something different on an img vs a canvas, b) it also seems that at least in Chrome how the scaling is done for images which are scaled down vs canvases is different.

A better solution would ideally meet these requirements:

• Attempt to not duplicate image data in memory any more than necessary (and also not unnecessarily utilize more CPU than is necessary - decoding in JS already requires a lot compared to native image handling)
• Have as much browser compatibility as feasible
• Use <img> tags instead of <canvas> (not a requirement, but would seem to be better)
• Provide an easy way to not only process images on document load, but also images that are added to the document later (e.g. in response to user activity)
• Still be simple to use (the existing technique on bellard.org is certainly easy to integrate)
• EDIT: Using web workers to decode the image without blocking the page is also potentially a good way to go.

Some relevant tools that come to mind include data: and blob: urls.

Anyone have examples of working code which loads BPGs using such "better" techniques? (The way Fabrice has it in his examples isn't bad, and certainly approaches have tradeoffs, but I think there may be something better for generalized use.)

Answer 1

BPG does look promising. If you want to detect the addition of <img> elements at any time, from any source, you can use MutationObservers. In case you don't know about them, they're asynchronous, logging a subset of all DOM mutations in a document and allowing the callback to handle those mutations at once rather than synchronously as with DOM Events. So if you create or change the source of a lot of images in a script, the observer will wait until your script finishes, then process all the new images in one go (hence the loops in the callbacks).

The following assumes that you have a function called doSomethingToDecode(img) (sorry I'm not going to help with that right now) that replaces the src of the img with (most likely) a generated PNG. It can do that asynchronously, that's not a problem. Also you don't need to stop observing the image while swapping its src as long as the generated remplacement doesn't end in ".bpg".

This first observer will react when you add <img> descendants anywhere (and other elements but it ignores those); unfortunately it is not possible to optimize it much in the general case. It has to iterate over the list of mutations, and then the list of new nodes for each mutation, hence those nested for loops. But

imgObs=new MutationObserver(
    function(mutations){
        for(var i=0, m; m=mutations[i]; i++) if(m.addedNodes.length)
            for(var j=0, img; img=m.addedNodes[j]; j++) if(img.localName=="img"){
                if(img.src && /\.bpg$/.test(img.src))
                    doSomethingToDecode(img)
                srcObs.observe(img, srcOptions)
            }
    }
)

This other observer reacts when you change the src attribute of an element, and should observe only <img> elements (for top performance, it has no failsafes in case you make it observe other elements, so don't).

srcObs=new MutationObserver(
    function(mutations){
        for(var i=0, m; m=mutations[i]; i++){
            var img=m.target
            if(img.src && /\.bpg$/.test(img.src))
                doSomethingToDecode(img)
        }
    }
)

Also keep this handy, we'll need it every time we start observing a new image:

var srcOptions={childList:false, attributes:true, attributeFilter:["src"]}

You could also make an observer that reacts to the removal of <img> elements to stop observing them then, and free any decoding-related ressources, but hopefully the browser is at least smart enough to stop observing elements that should be garbage collected (not tested!).

Run this after loading the HTML (don't wait for the whole page with images and CSS etc). Note: this is using the DOM level 0 document.images collection. Very old-school but it is doing exactly what we want very efficiently and concisely, so why the hell not?

So first you decode existing <img>s with bpg sources, and observe them for src changes later on:

for(var i=0, img; img=document.images[i]; i++){
    if(img.src && /\.bpg$/.test(img.src))
        doSomethingToDecode(img)
    srcObs.observe(img, srcOptions)
}

Then this tells the first observer to react to stuff in the entire document's <body>; unfortunately there is no tagNameFilter parameter to natively filter out non-<img> childList mutations.

imgObs.observe(document.body, {subtree:true, childList:true, attributes:false})