WebGL - How to render large texture?

Question

I'm trying to do some image processing in WebGL. But if I try to load large pictures (from camera - 8 MP) in a GLSL texture on mobile devices, the browser crashes. Small pictures working fine. So I think it's running out of memory.

I've googled a lot, but didn't find a solution. I think the best way would be to implement a "tile based" rendering. Splitting the 8 MP picture into smaller parts, render them and stick them together. But there will be problems, like applying a blur effect. You will see the edges of the "sub renderings". So I have to render overlapping pixels and "fade" them together. Didn't sound that good.

Are there any solutions I don't think of? How to work with really huge textures on mobile devices?

Answer 1

It's unlikely that one 8mp image is going to exceed the available ram. At RGB (byte) it would occupy 24mb of ram. Even at half floating point precision it'd be 50mb of ram. ARM systems use a unified ram architecture where the GPU and CPU share the ram. The iPhone for instance has 1gb of ram available.

Texturing support should be up to the desired size (for instance 3264x2248) which seems to be no problem on high end mobiles (4096x4096 supported by 81% of mobiles).

It's possible the maximum drawingbuffer size is smaller than the maximum supported texture size (you can query that).

Other than that, you might simply be running into a bug, which is annoying. So the way to deal with that is that you subdivide your image into smaller tiles and process each tile one at a time.

There are two ways you can deal with borders if you have filter kernels.

• You could pass in 9 textures and branch on texture lookup to address the correct one (that might be slow, some GPUs don't support branching and will execute all codepaths). Border textures would just be 2x2 so would have little ram footprint. Note that between textures there would be no interpolation, so don't rely on it, or substitute your own.
• You could add a half kernel sized border around each tile taking pixels from neighboring tiles in a preprocessing step.

Answer 2

What you need to do to get your tiles to work is provide enough input at the boundary to cover the non-locality of your image processing effects — e.g. the radius of your blur kernel. The simplest thing I can think of is:

For each tile of the output image, load the corresponding input tile and its 8 neighbors. Then run your blur filter using all 9 as input: where you would usually call texture2D, instead use a (custom) function which takes the coordinates and looks them up in the appropriate one of the 9 textures.

To use fewer textures, offset your output tiles compared to your input tiles by half their side length in both dimensions; then you only need to use 4 input tiles for each tile computation, but you will have an extra border in your output tiles if you use enough tiles to cover all of the input. (On the other hand, that can be a good thing if you are smart about trimming afterward; for example, if I apply a drop-shadow to an input image, I'd like to have the shadow enlarge the output rather than get truncated at the original boundary.)

If you do this correctly, there should be no artifacts at the tile boundaries.

(Disclaimer: While this solution seems simple, obvious, and correct to me, I am not an expert at either image processing or GPU resource conservation. There may be reasons why this doesn't actually work, or much better ideas out there.)