How to load textures in OpenGL ES efficiently

Question

I have a very basic knowledge of using OpenGL, especially on Android. I'm developing an app that uses OpenGL in order to switch between full screen images in a fast way (since it's too slow using the normal Android framework).

What I've found is that in order to load textures, I need to do something like:

ByteBuffer byteBuffer = ByteBuffer.allocateDirect(vertices.length * 4);
byteBuffer.order(ByteOrder.nativeOrder());
vertexBuffer = byteBuffer.asFloatBuffer();
vertexBuffer.put(vertices);
vertexBuffer.position(0);
byteBuffer = ByteBuffer.allocateDirect(texture.length * 4);
byteBuffer.order(ByteOrder.nativeOrder());
textureBuffer = byteBuffer.asFloatBuffer();
textureBuffer.put(texture);
textureBuffer.position(0);
_gl = gl;
final Bitmap bitmap = BitmapFactory.decodeResource(context.getResources(), _imageResourceId);
gl.glGenTextures(1, textures, 0);
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap, 0);
bitmap.recycle();

Now, since the images are meant to be full screen (and they are quite large - 1024x1024), this takes some time, especially since I need to load 5 of them at the same time.

So I need to ask some questions regarding tips of improving the code, especially about efficient loading (but also using less memory if possible):

1. if I decode the bitmap to not have transparency pixels, by using the RGB_565 format, will it boost the speed of loading the images to OpenGL (or the decoding phase)?

2. does the input bitmap have to have a width and height that are powers of 2 or can I let OpenGL take only the part that it wants, which will have this rule? I mean, maybe I can make the texImage2D command to take only a part of the bitmap?

3. maybe I can even decode only this part from the beginning (so if I have a huge image of 10000x10000, I can decode only a 1024x1024 part of it and give it to OpenGL)?

4. is it possible to load and show just the first image, and in the background load the rest? I've heard that you can't send the bitmaps to OpenGL in a thread that is not the one that handles it. Does it make sense to load them on the onDrawFrame method of the GlRenderer, say, the second time it get called?

5. I remember I've heard of a tip about merging multiple images into a single image (one after another, from left to right), so that it might have some speed boost for the decoding phase. Not sure what is the name of this technique. Is it still possible for OpenGL ES on Android?

6. is it possible to avoid creating a Bitmap instance and let the decoding done in a more native way (NDK , perhaps)? According to my tests, decoding a PNG file of size 1024x1024 on an emulator takes about 400ms-700ms, yet sending it to OpenGL takes about 50ms-70ms.

7. using Procrank, I've found out that OpenGL can take a lot of memory. Is it possible to make it use less memory? Maybe use better textures types?

8. since Android can work on many kinds of devices, is it possible to put in the manifest a requirement of how much memory is needed for the app to run, so that people with too little memory won't be able to install it?

---

@Majid Max :

1. so it's enough to decode this way and send it to OpenGL, or should I also set something special when sending to openGL?

2. there is no such a command to take a partial part of the bitmap?

3. I meant, can I decode only a partial part of the file to be stored in a bitmap, instead of all of the bitmap?

4. so the only thing I can do I to load everything in the beginning, and then use it all? How could it be? How do games handle it? I mean, they do show one stage after another, even with something that looks like an OpenGL generated progress bar. This is very problematic.

5. what I'm describing is available on the web too. For example, instead of loading multiple tiny images, the webpage contain a single image and maps what part of it should be shown where. Another name for it is sprites. Example here .

6. I see. I guess I should also call glDeleteTextures when I see there aren't used anymore, right?

7. how do I do that? What should I change in the code?

8. what happens when I use a lot of memory? Is there such a thing as virtual RAM (uses internal storage perhaps) when there is no free RAM? I've heard about it on Google IO video, but it seems like they weren't sure about the answer as well. Link here. They only said to expect more crashes when such a thing occurs.

---

@Majid Max :

1+2+3. ?

1. this might work. You mean that I make a new thread that will load the bitmaps, and then send the results back to the OpenGL thread which will bind the textures to the bitmap? Maybe I could use a similar approach to asyncTask, and use publishprogress.

2. that's also a good thing. Do you have any link I should read about it? Or maybe a snippet to change in my code?

3. thanks.

4. so i think it's the easiest thing to use ETC1. However it doesn't support transparency at all, so according to this link, I need to use another texture just for that, but I can't find a sample for this.

5. what can I assume about the available memory that I can use? Can I assume, for example, that all Android devices can offer me 200MB of video memory that I can use for OpenGL?

Answer 1

1. if I decode the bitmap to not have transparency pixels, by using the RGB_565 format, will it boost the speed of loading the images to OpenGL (or the decoding phase)?

   yes, if you don't need the transparency (alpha) use a format with no alpha, and smaller formats like (RGB565) means smaller size textures which certainly speed up the decoding and loading to opengl.

2. does the input bitmap have to have a width and height that are powers of 2 or can I let OpenGL take only the part that it wants, which will have this rule? I mean, maybe I can make the texImage2D command to take only a part of the bitmap?

   definitely YES, if you load a texture with non power of 2 width/height, opengl will allocate a texture with the next power of 2 (513/513 texture will become 1024/1024), and that means you are wasting vram memory. and you cant command opengl to take a part of the image because "teximage2d" takes width/height of the loaded image, and specifying any other values will give you a corrupted image (if not crushing the app).

3. maybe I can even decode only this part from the beginning (so if I have a huge image of 10000x10000, I can decode only a 1024x1024 part of it and give it to OpenGL)?

   no comment.

4. s it possible to load and show just the first image, and in the background load the rest? I've heard that you can't send the bitmaps to OpenGL in a thread that is not the one that handles it. Does it make sense to load them on the onDrawFrame method of the GlRenderer, say, the second time it get called?

   I'm not sure of that, there is already a multi-threaded rendering engine based on opengl. and you can load a texture (from other thread) while issuing a rendering command (at least in native C/C++, not sure about java). and NO, do not ever try to load a texture in rendering loop, that's bad practice.

5. I remember I've heard of a tip about merging multiple images into a single image (one after another, from left to right), so that it might have some speed boost for the decoding phase. Not sure what is the name of this technique. Is it still possible for OpenGL ES on Android?

   not sure what you mean here

6. is it possible to avoid creating a Bitmap instance and let the decoding done in a more native way (NDK , perhaps)? According to my tests, decoding a PNG file of size 1024x1024 on an emulator takes about 400ms-700ms, yet sending it to OpenGL takes about 50ms-70ms.

   this is the best way, the native code (C/C++) always better

7. using Procrank, I've found out that OpenGL can take a lot of memory. Is it possible to make it use less memory? Maybe use better textures types?

   (if you mean vram here) yes, using a texture compression format like ETC1, PVRTC, ATC always a good idea, it take less vram memory and yields a better performance.

8. since Android can work on many kinds of devices, is it possible to put in the manifest a requirement of how much memory is needed for the app to run, so that people with too little memory won't be able to install it?

   consider switching to a fallback solution (even more smaller textures) rather that chopping off lower-end devices.

EDIT:

4.. you start loading contents (images) in a separate thread which indicate to the rendering thread the percentage of loading process, this can be used to draw a progress bar for the loading process.

5.. this trick called "texture atlases" and it's for speeding up the rendering not the loading.

6.. after loading a decoded image into an opengl texture, you are free to delete the decoded image (from system memory) than you load the next one. and after you done using the opengl texture you can delete it (from video memory).

7.. texture compression is hardware-specific opengl extensions, so you have to check if the texture compression extension is present and then use it with "glCompressedTexImage2D" function instead of "texImage2D". PVRTC for PowerVR GPU; ATC for AMD GPU; ASTC for Mali GPU; ETC1 a standard texture format (supported in android 2.2+).

8.. when you done loading an image to opengl texture, the image is no longer needed. so you have to free the ram from unneeded contents (images).

EDIT2:

5.. http://http.download.nvidia.com/developer/NVTextureSuite/Atlas_Tools/Texture_Atlas_Whitepaper.pdf

7.. after you have loaded two ETC textures (ex: the first texture hold the color, and the second store alpha in the red channel) using "glCompressedTexImage2D" in opengl (lets say textureId1, textureId2), bind the two textures in two texture units :

glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, textureId1);

glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, textureId2);

and then create a shader program with the following fragment shader:

uniform sampler2D     sTexture1;
uniform sampler2D     sTexture2;
varying mediump vec2  vTexCoord;

void main()
{
    gl_FragColor.rgb = texture2D(sTexture1, vTexCoord).rgb;
    gl_FragColor.a   = texture2D(sTexture2, vTexCoord).r;
}

finally, bind the two shader texture samplers to the two textures units (which point to the two ETC textures):

GLint texture1Sampler = glGetUniformLocation(programObject, "sTexture1");
GLint texture2Sampler = glGetUniformLocation(programObject, "sTexture2");

glUniform1i(texture1Sampler, GL_TEXTURE0);
glUniform1i(texture2Sampler, GL_TEXTURE1);

8.. you cant assume any thing, you keep allocating textures and buffer (as needed) untill you get GL_OUT_OF_MEMORY, than you have to free other unused resources (textures, buffers, ...).