OpenGL/DirectX Hook - Similar to FRAPS

Question

Is it possible to detect what applications are using OpenGL or DirectX similar to what FRAPS does? (Possibly using some form of hook)? I probably won't need to actually draw to the window, I just need to know what processes are doing some form of 3D rendering for the time being.

(Edit:) In case you are not familiar with it, FRAPS is a program that can be used to draw a "Frame-per-second" counter on a 3D application. FRAPS finds all running 3D applications by itself without needing you to specify the process name.

Example of "Frame Per second" counter drawn to external game:

Answer 1

Probably the simplest way is to check for the presence of the OpenGL and DirectX core libraries, probably also a good idea to add in the driver OGL dlls in too (such as nvogl), this can be done via EnumProcesses & EnumProcessModulesEx, using p/invoke, this will at least give you a starting set of processes possibly using OGL or DX.

Of course some applications load both of the API's and use only one, or only conditionally use one of the GFX API's (though the latter only occurs with specialized tools and the like), for this, IMO, the best way to check is to perform some form of injection or attaching to the process like a debugger would, then hooking either Present for DX or wglSwapBuffers for OGL.

You might be about to get away with not using a hook by enumerating the GDI handles and looking for the DXGI or OGL render contexts, how viable this is, I don't know.

Answer 2

From what I understand, FRAPS uses a relatively brute force approach to determining where to lay down shop. The process gets started with SetWindowsHookEx to request that the OS load the FRAPS hook DLL into every running process it can [and future processes]. The magic in the DLL comes down to running a procedural set of tests using GetModuleHandleA to observe if the process it is attached to has loaded any OpenGL/DirectX modules. If all calls return NULL, the hook attempts to remove itself from the process.

On the other hand, if the process has loaded them, it simply hooks the appropriate rendering function from that library by removing protection and injecting a JMP hook. wglSwapBuffers is typically the only relevant one in OpenGL. When the process calls this function, it ends up calling the FRAPS module and then FRAPS captures the back buffer into its queue for encoding to AVI and renders its little indication. Then it processes the original request for wglSwapBuffers and returns the execution back to the program.

As far as querying in C#... review EasyHook (http://easyhook.codeplex.com/) and see if it doesn't work for you. I personally have no experience with this API.