Invalid blending results across all browsers with HTML5 canvas

Question

Summary

When repeatedly drawing anything (apparently with a low alpha value) on a canvas, regardless of if it's with drawImage() or a fill function, the resulting colors are significantly inaccurate in all browsers that I've tested. Here's a sample of the results I'm getting with a particular blend operation:

Problem Demonstration

For an example and some code to play with, check out this jsFiddle I worked up:

http://jsfiddle.net/jMjFh/2/

The top set of data is a result of a test that you can customize by editing iters and rgba at the top of the JS code. It takes whatever color your specify, RGBA all in range [0, 255], and stamps it on a completely clean, transparent canvas iters number of times. At the same time, it keeps a running calculation of what the standard Porter-Duff source-over-dest blend function would produce for the same procedure (ie. what the browsers should be running and coming up with).

The bottom set of data presents a boundary case that I found where blending [127, 0, 0, 2] on top of [127, 0, 0, 63] produces an inaccurate result. Interestingly, blending [127, 0, 0, 2] on top of [127, 0, 0, 62] and all [127, 0, 0, x] colors where x <= 62 produces the expected result. Note that this boundary case is only valid in Firefox and IE on Windows. In every other browser on every other operating system I've tested, the results are far worse.

In addition, if you run the test in Firefox on Windows and Chrome on Windows, you'll notice a significant difference in the blending results. Unfortunately we're not talking about one or two values off--it's far worse.

Background Information

I am aware of the fact that the HTML5 canvas spec points out that doing a drawImage() or putImageData() and then a getImageData() call can show slightly varying results due to rounding errors. In that case, and based on everything I've run into thus far, we're talking about something miniscule like the red channel being 122 instead of 121.

As a bit of background material, I asked this question a while ago where @NathanOstgard did some excellent research and drew the conclusion that alpha premultiplication was to blame. While that certainly could be at work here, I think the real underlying issue is something a bit larger.

The Question

Does anyone have any clue how I could possibly end up with the (wildly inaccurate) color values I'm seeing? Furthermore, and more importantly, does anyone have any ideas how to circumvent the issue and produce consistent results in all browsers? Manually blending the pixels is not an option due to performance reasons.

Thanks!

---

Edit: I just added a stack trace of sorts that outputs each intermediate color value and the expected value for that stage in the process.

---

Edit: After researching this a bit, I think I've made some slight progress.

Reasoning for Chrome14 on Win7

In Chrome14 on Win7 the resulting color after the blend operation is gray, which is many values off from the expected and desired reddish result. This led me to believe that Chrome doesn't have an issue with rounding and precision because the difference is so large. Instead, it seems like Chrome stores all pixel values with premultiplied alpha.

This idea can be demonstrated by drawing a single color to the canvas. In Chrome, if you apply the color [127, 0, 0, 2] once to the canvas, you get [127, 0, 0, 2] when you read it back. However, if you apply [127, 0, 0, 2] to the canvas twice, Chrome gives you [85, 0, 0, 3] if you check the resulting color. This actually makes some sense when you consider that the premultiplied equivalent of [127, 0, 0, 2] is [1, 0, 0, 2].

Apparently, when Chrome14 on Win7 performs the blending operation, it references the premultiplied color value [1, 0, 0, 2] for the dest component and the non-premultiplied color value [127, 0, 0, 2] for the source component. When these two colors are blended together, we end up with [85, 0, 0, 3] using the Porter-Duff source-over-dest approach, as expected.

So, it seems like Chrome14 on Win7 inconsistently references the pixel values when you work with them. The information is stored in a premultiplied state internally, is presented back to you in non-premultiplied form, and is manipulated using values of both forms.

I'm thinking it might be possible to circumvent this by doing some additional calls to getImageData() and/or putImageData(), but the performance implications don't seem that great. Furthermore, this doesn't seem to be the same issue that Firefox7 on Win7 is exhibiting, so some more research will have to be done on that side of things.

---

Edit: Below is one possible approach that seems likely to work.

Thoughts on a Solution

I haven't gotten back around to working on this yet, but one WebGL approach that I came up with recently was to run all drawing operations through a simple pixel shader that performs the Porter-Duff source-over-dest blend itself. It seems as though the issue here only occurs when interpolating values for blending purposes. So, if the shader reads the two pixel values, calculates the result, and writes (not blends) the value into the destination, it should mitigate the problem. At the very least, it shouldn't compound. There will definitely still be minor rounding inaccuracies, though.

I know in my initial question I mentioned that manually blending the pixels wouldn't be viable. For a 2D canvas implementation of an application that needs real-time feedback, I don't see a way to fix this. All of the blending would be done on the CPU and would prevent other JS code from executing. With WebGL, however, your pixel shader runs on the GPU so I'm pretty sure there shouldn't be any performance hit.

The tricky part is being able to feed the dest canvas in as a texture to the shader because you also need to render it on a canvas to be viewable. Ultimately, you want to avoid having to generate a WebGL texture object from your viewable canvas every time it needs to be updated. Maintaining two copies of the data, with one as an in-memory texture and one as a viewable canvas (that's updated by stamping the texture on as needed), should solve this.

Answer 1

Oh geez. I think we just stepped into the twilight zone here. I'm afraid I don't have an answer but I can provide some more information at least. Your hunch is right that at least something is larger here.

I'd prefer an example that is drastically more simple (and thus less prone to any sort of error - I fear you might have a problem with 0-255 somewhere instead of 0-1 but didnt look thoroughly) than yours so I whipped up something tiny. What I found wasn't pleasant:

<canvas id="canvas1" width="128" height="128"></canvas>
<canvas id="canvas2" width="127" height="127"></canvas>

+

var can = document.getElementById('canvas1');
var ctx = can.getContext('2d');
var can2 = document.getElementById('canvas2');
var ctx2 = can2.getContext('2d');

var alpha = 2/255.0;

ctx.fillStyle = 'rgba(127, 0, 0, ' + alpha + ')';
ctx2.fillStyle = 'rgba(127, 0, 0, ' + alpha + ')'; // this is grey
//ctx2.fillStyle = 'rgba(171, 0, 0, ' + alpha + ')'; // this works

for (var i = 0; i < 32; i++) {
      ctx.fillRect(0,0,500,500);
      ctx2.fillRect(0,0,500,500);
}

Yields this:

See for yourself here:

http://jsfiddle.net/jjAMX/

It seems that if your surface area is too small (not the fillRect area, but the canvas area itself) then the drawing operation will be a wash.

It seems to work fine in IE9 and FF7. I submitted it as a bug to Chromium. (We'll see what they say, but their typical M.O. so far is to ignore most bug reports unless they are security issues).

I'd urge you to check your example - I think you might have a 0-255 to 0-1 issue somewhere. The code I have here seems to work just dandy, save for maybe the expected reddish color, but it is a consistent resultant color across all browsers with the exception of this Chrome issue.

In the end Chrome is probably doing something to optimize canvases with a surface area smaller than 128x128 that is fouling things up.