How can I debug an ActiveX control (OCX), or make it log errors?

Question

I'm currently working with a rather old Borland C++ application, which is using an ActiveX component to draw some graphics. In the application multiple windows with the ActiveX comp. can be open at any time - these can either show the same graphics (different zoom factor etc), or different graphics.

The application is for positioning, and the ActiveX is drawing and showing positions of different units.

Around 10 times a second the Borland application got a new position, and finds out which forms (and their ActiveX) needs to know about the updated position in order to draw it. This have been going well for a long time, but I have had to do quite some changes in the ActiveX for a new version of the product.

About a year ago I had to do some minor changes in the component aswell, and I found that the application could end in a state, causing an "index out of bounds" error in the component. The result of this was not an error getting shown or the program terminating, but instead the application started to use a huge amount of memory - and kept going up very fast. At some point it stopped and the component that had the error simply stopped showing anything (stopped drawing itself).

Now with the recent changes I have made, I have the same problem where one of the components seems to get an error, which is not getting shown, and instead it is not redrawing itself, and the memory use is going sky-high. On some PC's it seems that an Access Violation is thrown - which is saying that the error happened in the OCX, but on the PC I develop on, I cant get this Access Violation in any way.

Also I cannot track down exactly when the error happens - ie what is causing the error. I can run the same setup 10 times in a row for 15 minutes, sometimes it happens that the memory use go up and a component bug, other times nothing happens and it runs as it should for the whole duration.

As it is an OCX, it is registered using regsvr32, and is therefore code-wise not part of the main application. Therefore I can not use breakpoints and debug it that way.

I am pretty sure that some error is happening inside the component, which is not passed on, so I cannot see what it is.

So do anyone know how I can debug this? Can I somehow make the OCX log any errors that happens, or make it show the error, or what can I do?

Any help would be much appreciated - been trying to track down the error for 3 days now with no result what so ever.

Answer 1

Essentially you are asking how to debug a DLL. An OCX is just a DLL file that's loaded into a process. This is a somewhat broad topic, but I'll try to give a brief start:

DLL / EXE / OCX files are usually called "modules" in the context of Windows programming. They are all basically the same thing. I will call them DLLs here though just for the sake of clarity.

Debuggers (Visual Studio and Borland are both debuggers as well as IDEs) "attach" like a parasite to processes, allowing you to do things like set breakpoints, read process memory, see stack trace, etc. They can see / manipulate all the memory & resources for that process including all DLLs.

DLLs do not contain much information to help debuggers, even in a debug build. They basically just contain binary machine code and if you step into a DLL call with the debugger, you'll only be able to see the assembly code -- not the original source code. Functions are just addresses in memory, local variables are not even visible; you only get some pointers into stack memory.

PDB files ("program database") contain all the additional information and metadata for the debugger to do things like mapping addresses in memory to lines of source code, local variables, datatypes, function signatures, etc. This information is called "debugging symbols" or just "symbols". When Visual Studio builds a DLL, it outputs a corresponding PDB file. It's this PDB file that enables all the magic of stepping through your source code in the debugger, viewing local variables, viewing datatypes properly in the watch window.

When Visual Studio's debugger is attached to a process and sees a DLL being loaded, it searches for its corresponding PDB file. It looks for this in a number of places - the simplest of which is in the same folder as the DLL. So if you loaded C:\something\myctl.ocx, it will look for C:\something\myctl.pdb. If it can find it, it will use it and you can debug the DLL with rich debugger support. If it can't find it, you will be where you are now - where DLL calls are a black box you cannot see into.

Microsoft even provides PDB files for Windows DLLs like ntdll.dll. They must be downloaded as needed. Visual Studio can do this automatically by going to Tools -> Options -> Debugging -> Symbols and there should be an option to use Microsoft Symbol Servers to automatically fetch missing symbol files.

Small example to put you in the right direction:

Let's say you wrote an OCX called myctl.ocx that crashes when it's added to a Wordpad document. The way to debug this is to attach the debugger to wordpad.exe. In Visual Studio that's Debug -> Attach to Process I believe. When it's attached, you can even see in the output window:

'wordpad.exe': Loaded 'C:\Program Files\Windows NT\Accessories\wordpad.exe', Symbols loaded (source information stripped).
'wordpad.exe': Loaded 'C:\Windows\System32\ntdll.dll', Symbols loaded (source information stripped).
'wordpad.exe': Loaded 'C:\Windows\System32\kernel32.dll', Symbols loaded (source information stripped).
'wordpad.exe': Loaded 'C:\Windows\System32\KernelBase.dll', Symbols loaded (source information stripped).
...

You can see how Visual Studio loads the PDB files (symbol files) that provide a bit of extra information for these. When you load myctl.ocx, you will see that line as well. If myctl.pdb is accessible, it will load that as well.

'wordpad.exe': Loaded 'C:\something\myctl.ocx', Symbols loaded.

With this, you can debug anything in myctl.ocx with source code and everything. When Wordpad crashes inside of myctl.ocx, it should show you the source code and everything, again assuming it's in an accessible location.