How can I draw (as in GLPaint) onto a background image, and with temporary drawings?

Question

I am writing a GLPaint-esque drawing application for the iPad, however I have hit a stumbling block. Specifically, I am trying to implement two things at the moment:

1) A background image that can be drawn onto.

2) The ability to draw temporary shapes, e.g. you might draw a line, but the final shape would only be committed once the finger has lifted.

For the background image, I understand the idea is to draw the image into a VBO and draw it right before every line drawing. This is fine, but now I need to add the ability to draw temporary shapes... with kEAGLDrawablePropertyRetainedBacking set to YES (as in GLPaint) the temporary are obviously not temporary! Turning the retained backing property to NO works great for the temporary objects, but now my previous freehand lines aren't kept.

What is the best approach here? Should I be looking to use more than one EAGLLayer? All the documentation and tutorials I've found seem to suggest that most things should be possible with a single layer. They also say that retained backing should pretty much always be set to NO. Is there a way to work my application in such a configuration? I tried storing every drawing point into a continually expanding vertex array to be redrawn each frame, but due to the sheer number of sprites being drawn this isn't working.

I would really appreciate any help on this one, as I've scoured online and found nothing!

Answer 1

I've since found the solution to this problem. The best way appears to be to use custom framebuffer objects and render-to-texture. I hadn't heard of this before asking the question, but it looks like an incredibly useful tool for the OpenGLer's toolkit!

For those that may be wanting to do something similar, the idea is that you create a FBO and attach a texture to it (instead of a renderbuffer). You can then bind this FBO and draw to it like any other, the only difference being that the drawings are rendered off-screen. Then all you need to do to display the texture is to bind the main FBO and draw the texture to it (using a quad).

So for my implementation, I used two different FBOs with a texture attached to each - one for the "retained" image (for freehand drawing), and the other for the "scratch" image (for temporary drawings). Each time a frame is rendered, I first draw a background texture (in my case I just used the Texture2D class), then draw the retained texture, and finally the scratch texture if required. When drawing a temporary shape everything is rendered to the scratch texture, and this is cleared at the start of every frame. Once it is finished, the scratch texture is drawn to the retained texture.

Here are a few snippets of code that might be of use to somebody:

1) Create the framebuffers (I have only shown a couple here to save space!):

// ---------- DEFAULT FRAMEBUFFER ---------- //
// Create framebuffer.
glGenFramebuffersOES(1, &viewFramebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, viewFramebuffer);

// Create renderbuffer.
glGenRenderbuffersOES(1, &viewRenderbuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, viewRenderbuffer);

// Get renderbuffer storage and attach to framebuffer.
[context renderbufferStorage:GL_RENDERBUFFER_OES fromDrawable:layer];
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, viewRenderbuffer);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight);

// Check for completeness.
status = glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES);
if (status != GL_FRAMEBUFFER_COMPLETE_OES) {
    NSLog(@"Failed to make complete framebuffer object %x", status);
    return NO;
}

// Unbind framebuffer.
glBindFramebufferOES(GL_FRAMEBUFFER_OES, 0);


// ---------- RETAINED FRAMEBUFFER ---------- //
// Create framebuffer.
glGenFramebuffersOES(1, &retainedFramebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, retainedFramebuffer);

// Create the texture.
glColor4f(0.0f, 0.0f, 0.0f, 0.0f);
glGenTextures(1, &retainedTexture);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, retainedTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 1024, 1024, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glBindTexture(GL_TEXTURE_2D, 0);

// Attach the texture as a renderbuffer.
glFramebufferTexture2DOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_TEXTURE_2D, retainedTexture, 0);

// Check for completeness.
status = glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES);
if (status != GL_FRAMEBUFFER_COMPLETE_OES) {
    NSLog(@"Failed to make complete framebuffer object %x", status);
    return NO;
}

// Unbind framebuffer.
glBindFramebufferOES(GL_FRAMEBUFFER_OES, 0);

2) Draw to the render-to-texture FBO:

// Ensure that we are drawing to the current context.
[EAGLContext setCurrentContext:context];
glBindFramebufferOES(GL_FRAMEBUFFER_OES, retainedFramebuffer);
glViewport(0, 0, 1024, 1024);

// DRAWING CODE HERE

3) Render the various textures to the main FBO, and present:

glBindFramebufferOES(GL_FRAMEBUFFER_OES, viewFramebuffer);
glViewport(0, 0, backingWidth, backingHeight);
glClearColor(1.0f, 1.0f, 1.0f, 1.0f);       // Clear to white.
glClear(GL_COLOR_BUFFER_BIT);


glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);

[self drawBackgroundTexture];
[self drawRetainedTexture];
[self drawScratchTexture];

glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);


glBindRenderbufferOES(GL_RENDERBUFFER_OES, viewRenderbuffer);
[context presentRenderbuffer:GL_RENDERBUFFER_OES];

For example, drawing drawing the retained texture using [self drawRetainedTexture] would use the following code:

// Bind the texture.
glBindTexture(GL_TEXTURE_2D, retainedTexture);

// Destination coords.
GLfloat retainedVertices[] = {
    0.0,          backingHeight,    0.0,
    backingWidth, backingHeight,    0.0,
    0.0,          0.0,              0.0,
    backingWidth, 0.0,              0.0
};

// Source coords.
GLfloat retainedTexCoords[] = {
    0.0, 1.0,
    1.0, 1.0,
    0.0, 0.0,
    1.0, 0.0
};

// Draw the texture.
glVertexPointer(3, GL_FLOAT, 0, retainedVertices);
glTexCoordPointer(2, GL_FLOAT, 0, retainedTexCoords);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

// Unbind the texture.
glBindTexture(GL_TEXTURE_2D, 0);

A lot of code, but I hope that helps somebody. It certainly had me stumped for a while!