Speed up HTML5 Canvas pixel rendering

Question

I am designing a Photoshop-style web application running on the HTML5 Canvas element. The program runs well and is very speedy until I add blend modes into the equation. I achieve blend modes by merging each canvas element into one and combining each pixel from each canvas using the right blend modes starting from the bottom canvas.

for (int i=0; i<width*height*4; i+=4) {
    var base = [layer[0][i],layer[0][i+1],layer[0][i+2],layer[0][i+3]];
    var nextLayerPixel = [layer[1][i],layer[1][i+1],layer[1][i+2],layer[1][i+3]];
    //Apply first blend between first and second layer
    basePixel = blend(base,nextLayerPixel);
    for(int j=0;j+1 != layer.length;j++){
        //Apply subsequent blends here to basePixel
        nextLayerPixel = [layer[j+1][i],layer[j+1][i+1],layer[j+1][i+2],layer[j+1][i+3]];
        basePixel = blend(basePixel,nextLayerPixel);
   }
   pixels[i] = base[0];
   pixels[i+1] = base[1];
   pixels[i+2] = base[2];
   pixels[i+3] = base[3];
}
canvas.getContext('2d').putImageData(imgData,x,y);

With it calling blend for different blend modes. My 'normal' blend mode is as follows:

var blend = function(base,blend) {
    var fgAlpha = blend[3]/255;
    var bgAlpha = (1-blend[3]/255)*base[3]/255;
    blend[0] = (blend[0]*fgAlpha+base[0]*bgAlpha);
    blend[1] = (blend[1]*fgAlpha+base[1]*bgAlpha);
    blend[2] = (blend[2]*fgAlpha+base[2]*bgAlpha);
    blend[3] = ((blend[3]/255+base[3])-(blend[3]/255*base[3]))*255;
    return blend;
}

My test results in Chrome (yielding some of the best out of the tested browsers) was around 400ms blending three layers together on a canvas 620x385 (238,700 pixels).

This is a very small implementation as most projects will be larger in size and include more layers which will make the execution time skyrocket under this method.

I'm wondering if there is any faster way to combine two canvas contexts with a blend mode without having to go through every pixel.

Answer 1

Don't create so many 4-value-arrays, it should go much faster when using the existent memory. Also, you might want to use the reduce function on your layer array, this seems exactly what you need. However, using no functions at all might be another touch faster - no creation of execution contexts needed. The following code will invoke the blend function only for each layer, not each pixel * layers.

var layer = [...]; // an array of CanvasPixelArrays
var base = imgData.data; // the base CanvasPixelArray whose values will be changed
                         // if you don't have one, copy layer[0]
layer.reduce(blend, base); // returns the base, on which all layers are blended
canvas.getContext('2d').putImageData(imgData, x, y);

function blend(base, pixel) {
// blends the pixel array into the base array and returns base
    for (int i=0; i<width*height*4; i+=4) {
        var fgAlpha = pixel[i+3]/255,
            bgAlpha = (1-pixel[i+3]/255)*fgAlpha;
        base[i  ] = (pixel[i  ]*fgAlpha+base[i  ]*bgAlpha);
        base[i+1] = (pixel[i+1]*fgAlpha+base[i+1]*bgAlpha);
        base[i+2] = (pixel[i+2]*fgAlpha+base[i+2]*bgAlpha);
        base[i+3] = ((fgAlpha+base[i+3])-(fgAlpha*base[i+3]))*255;
//                           ^ this seems wrong, but I don't know how to fix it
    }
    return base;
}

---

Alternative solution: Don't blend the layers together in javascript at all. Just absolutely position your canvases over each other and give them a CSS opacity. This should speed up the displaying a lot. Only I'm not sure whether this will work together with your other effects, should they need to be applied on multiple layers.

Answer 2

Traditionally these type of massive pixel manipulation is sped up by running them on the GPU, rather than on the CPU. Unfortunately canvas doesn't have support for this but you could potentially implement a workaround using SVG Filters. This would allow you to use hardware accelerated blend modes (feBlend) to blend two images together. If you render your layers to two images and then refer these images in your SVG you could make this work.

Here is a nice illustrated overview how this could work:

http://blogs.msdn.com/b/ie/archive/2011/10/14/svg-filter-effects-in-ie10.aspx (for IE10 but applies to any browser which supports SVG Filters)