Injecting a Frame Buffer

Question

I've been "hacking" one of my favorite games from when I was younger, and I've managed to be able to intercept OpenGL calls and inject C++ code using an opengl32.dll wrapper. I've been able to come up with a number of cool uses for this, but my current goal is to implement post processing glsl shaders into this old game to make it look a bit more modern. The game was originally released in the late 90's, so its usage of OpenGL and its rendering code is limited/primitive/outdated.

I have been able to successfully produce vertex and fragment shaders by injecting code to initialize and use glsl shader programs, but my problem lies in producing a framebuffer / rendering my scene to a texture to send to my fragment shader as a sampler. As far as I can tell, the texture I produce is usually all black.

It's hard to find resources that are relevant to what I am doing and examining the raw opengl calls (I have been trying to determine where to insert the code by reading an opengl trace of a single frame of gameplay), so I haven't been able to wrap my head around what I'm doing wrong.

I've tried putting the code in many places, but currently, I'm binding the framebuffer after the game calls wglSwapBuffers() and I'm unbinding it immediately before the next glFlush() call is made, but I'm not sure if the multiple viewports setup, or matrix operations, or whatever else created between these times is somehow messing with the framebuffer.

This is the code to initialize the framebuffer:

void initFB()
{
  /* Texture */
  glActiveTexture(GL_TEXTURE0);
  glGenTextures(1, &fbo_texture);
  glBindTexture(GL_TEXTURE_2D, fbo_texture);

  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 1920, 1080, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);

  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

  glBindTexture(GL_TEXTURE_2D, 0);

    /* Depth buffer */
  glGenRenderbuffers(1, &fbo_depth);
  glBindRenderbuffer(GL_RENDERBUFFER, fbo_depth);
  glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, 1920, 1080);  
  glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, fbo_depth);
  glBindRenderbuffer(GL_RENDERBUFFER, 0);

  /* Framebuffer to link everything together */
  glGenFramebuffers(1, &fbo);
  glBindFramebuffer(GL_FRAMEBUFFER, fbo);

  glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, fbo_texture, 0);
  glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, fbo_depth);

  GLenum status;
  if ((status = glCheckFramebufferStatus(GL_FRAMEBUFFER)) != GL_FRAMEBUFFER_COMPLETE) {
    fprintf(stderr, "glCheckFramebufferStatus: error %p", status);
  }

  glBindFramebuffer(GL_FRAMEBUFFER, 0);
}

This is the code called after wglSwapBuffers:

GLenum fboBuff[] =  { GL_COLOR_ATTACHMENT0 };
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glDrawBuffers(1, fboBuff);
glPushAttrib(GL_VIEWPORT_BIT | GL_ENABLE_BIT);

This is the code called before glFlush():

glBindFramebuffer(GL_FRAMEBUFFER, 0);
glPopAttrib(); // Restore our glEnable and glViewport states  

... Draw an Overlay containing with fbo texture binded ...

Here is the gl trace of a single frame.

It's nearly 800 lines but a trained opengl eye should be able to see what calls are getting made when. I removed several hundred lines of drawing calls that I atleast understand.

Answer 1

I am not completely sure what you want to do. Run the game to render to you fbo? Render stuff on top? To start, try hooking all wgl and gl calls. http://research.microsoft.com/en-us/projects/detours/ is a good place to start.

But reading your post again, I guess you already know that. It would help to know what you want to do.

• Rendering an overlay: Hook SwapBuffers, do your stuff there
• Capturing: Again, hook SwapBuffers
• Rendering to texture and doing post effects: Hook BindFramebuffer. But there is a catch: frame buffer 0 is the default at startup, so code that does not use it will not call BindFramebuffer to start, only to set it back to 0. Try hooking glClear or whatever call comes first.

Feel free to pm me with more details. Sounds like a fun educational project I would love to help with :)

I just read your post again before posting, and your problem might be a lot easier. As a windows program your target assumes that fbo 0 is bound when it starts rendering a frame. Just do that. Before it's done, it'll switch back to fbo 0. Don't do the push/pop thing. Just make sure that your fbo is the same size (on windows bigger is ok also) it expects. You need to hook BindFramebuffer though. Whenever your game calls bind 0, you need to bind yours instead. But there is no need to do anything else like touching the viewport, the game has to take care of that already. When you finally render in your swapBuffers hook, just make sure to restore anything you change.

Also you really don't need the DrawBuffers call. It confused the mole out of me and probably also of your target. :) Your target does GL 1.2 code, you are doing 3...4...

Answer 2

From my experience, the things you are trying to do are quite possible with an GL interception approach for a shaderless GL1.x application.

As I quick hack, you could just forget the FBO for now and just use glCopyTexSubImage() when ever the original application swaps the buffers, to copy the rendered content into a texture and text your postprocessing functions.

Some things you should take care of, when doing such an GL injection approach:

• there might be different GL contexts (although this is more likely for CAD applications than typical games)
• old GL allowed the user to directly manage Object ID - no calls to glGen* functions where needed (Nice applications still did them). If you yourself create new objects, the original app will not know about and might reuse those IDs. If you want to get this right, you would have to transparently remap Object IDs (possibly using some hash table) - which is a lot of work, as many GL functions are involved. Another evil hack would be to use some unlikely Object IDs yourself.
• GL is a state machine, so you should be very careful that you restore the state the game is expecting (as much as possible). OTOH hand, you should also be prepared that the original application might have any GL state set when you got the change to intercept it - so you might have to reset a lot of things.

I can assure you that it is indeed possible to inject rendering to texture into some old GL application, as I did it myself. Worked very nice with Quake3 and other famous games of the times. I don't see an obviuos error with your current approach. I few things though: You should also intercept glDrawBuffer() and relay any drawing the app is going to do to GL_BACK (assuming double-buffered application here) to your color attachment. The log you posted has not trace of this function though, so the game might have set it once at startup.

I also do not understand why you use glFlush. It is not a good marker for anything in particular. If the game uses double-buffering, SwapBuffers will be the best approximation of a "frame" you can get. Just insert your own init object creations stuff directly after context creation (you can intercept wglCreateCintext() and friends, too), and already set up rendering to the FBO. OnSwapBuffers, just draw the contents of the texture using your shaders to the real back buffer, do the real buffer swap, and restore the render to FBO setup.

I don't know if you are already aware of it or not, but the open source project Chromium does offer a nice fframework for OpenGL interception. While the focus of this might be on distributed cluster rendering, it will work also in alocal setup and allows you to write SPUs (stream processing units), where you can simply sppecify a C function for any of the GL functions you want to hook on. The project is outdated and not in active develpoment since 5 years or so, because the kind of GL stream manipulation it was desgined to ist not going to work with the more modern programmable pipeline, but it will still work well with the good old GL1.x fixed-function pipeline...