Can I switch the Visual C++ runtime to another heap?

Question

My program uses a third party dynamic link library that has huge memory leaks inside. Both my program and the library are Visual C++ native code. Both link to the Visual C++ runtime dynamically.

I'd like to force the library into another heap so that all allocations that are done through the Visual C++ runtime while the library code is running are done on that heap. I can call HeapCreate() and later HeapDestroy(). If I somehow ensure that all allocations are done in the new heap I don't care of the leaks anymore - they all go when I destroy the second heap.

Is it possible to force the Visual C++ runtime to make all allocations on a specified heap?

Answer 1

Sorry my last answer got posted half-baked, i pressed tab and enter without remembering this was a text box and not an editor...

Anyway heres it in full :

You can use the detours library to hook the allocation and deallocation functions, replace them with your own :

Vaguely something like this :

//declare a global 
HANDLE g_currentHeap;

LPVOID WINAPI HeapAlloc(HANDLE hHeap, DWORD dwFlags, SIZE_T dwBytes) 
{ 
    return OriginalHeapAlloc(g_currentHeap, dwFlags, dwBytes);
}


BOOL WINAPI HeapFree(HANDLE hHeap, DWORD dwFlags, LPVOID lpMem)
{
    return OriginalHeapFree(g_currentHeap, dwFlags, lpMem);
}

in the application load

HANDLE g_Heaps[2];

int main()
{
    // Two heaps
    g_Heaps[0] = HeapCreate(...);
    g_Heaps[1] = HeapCreate(...);


    // Do whatevers needed to hook HeapAlloc and HeapFree and any other heap functions 
    // and redirect them to the versions above
    // Save the old function pointers so we can call them
}

Then everytime you call an API from the 3rd party DLL you can do this

void someFn()
{
    g_currentHeap = g_Heaps[1];
    Some3rdPartyAPI();
    g_currentHeap = g_Heaps[0];

    SomeOtherFunction();

}

This should solve your problem

@peterchen : The C++ runtime calls HeapAlloc for new and malloc() so this approach will work. In fact I believe almost any languages runtime will use the win32 Heap functions, unless there was a special reason not to.