How to render to a GL_TEXTURE_EXTERNAL_OES?

Question

I need a way to render to a GL_TEXTURE_EXTERNAL_OES texture. I tried binding the texture to a framebuffer but I got GL_INVALID_ENUM error. Here is some sample code:

glEnable(GL_TEXTURE_EXTERNAL_OES);
glGenFramebuffersOES(1, &frameBuffer);
glBindFramebufferOES(GL_FRAMEBUFFER, frameBuffer); 
glFramebufferTexture2DOES(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_EXTERNAL_OES, outTexHandle, 0); // I get GL_INVALID_ENUM here
// set viewport, uniforms and draw 
glBindFramebufferOES(GL_FRAMEBUFFER, 0);
glDisable(GL_TEXTURE_EXTERNAL_OES);

My final goal is to modify the camera frame using SurfaceTexture by applying some filters on it. Therefore, my shader program has both as input and output the same GL_TEXTURE_EXTERNAL_OES texture. Is this possible? If not are there any workarounds? For my application it is essential that the output texture is GL_TEXTURE_EXTERNAL_OES, I cant't use a GL_TEXTURE_2D texture.

Answer 1

You cannot render to a GL_TEXTURE_EXTERNAL_OES texture. GL_TEXTURE_EXTERNAL_OES is the external texture which is rendered to by other devices such as the devices camera. You have read access to this texture. If you need to change how the texture is rendered then you will need to write a shader program to do so.

There are a number of steps you must perform before you can access the OES texture data. The external OES texture must be initialised in the following way:

int[] mTextureHandles = new int[1];
GLES20.glGenTextures(1, mTextureHandles, 0);
mTextureHandle = mTextureHandles[0];

GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, mTextureHandles[0]);
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

You then wrap OES texture in a SurfaceTexture object and set it to update for each frame. Your class must implement the SurfaceTexture.OnFrameAvailableListener interface:

camTexture = new SurfaceTexture(mTextureHandle);
camTexture.setOnFrameAvailableListener(this);

When initialising the camera, you must point it to use the GL_TEXTURE_EXTERNAL_OES like so:

mCamera = Camera.open();
try{
    mCamera.setPreviewTexture(camTexture);
}catch(IOException ioe){
    ioe.printStackTrace();
}

Now in your public void onFrameAvailable(SurfaceTexture surfaceTexture) callback, you render to one of the GL_TEXTURE objects like so:

GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
camTexture.updateTexImage();
renderFrame();

In your renderFrame() method you need to do the following:

1 - Set viewport

2 - Bind to the external texture: glBindTexture(GL_TEXTURE_EXTERNAL_OES, texId);

3 - Set all of the variables for the shader program including transformation matrices

4 - Finally render the texture with a call to glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

If your goal is to change the way that the texture is rendered, you must implement a GLES shader program to do so. Here is an example of a simple texture vertex shader:

uniform mat4 uMVPMatrix;
attribute vec4 aPosition;
attribute vec4 aTextureCoord;
varying vec2 vTextureCoord;
void main() {
    gl_Position = uMVPMatrix * aPosition;
    vTextureCoord = vec2(1.0 - aTextureCoord.s, aTextureCoord.t);
}

And here is the associated fragment shader:

#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES sTexture;
varying vec2 vTextureCoord;
void main(){
    gl_FragColor = texture2D(sTexture, vTextureCoord);
}

Note that the fragment shader has the #extension GL_OES_EGL_image_external : require at the top of the shader file. This is very important, it is required to allow the shader to read from the GL_TEXTURE_EXTERNAL_OES texture. You can now change the shader program to render the external texture data any way that you wish.