How I can limit the heap's size so when I allocate a lot it won't get the machine stuck?

Question

I have to debug program that rapidly allocates memory sometimes (Not by design.) and when it happens my whole computer just stop responding because physical memory goes 100% (I have 4GB ram), then I have to press the restarting button everytime with no way to know why did it happen.

Is there a way to limit new's or malloc's heap's size? By limiting I mean that it will throw exception like C#'s OutOfMemoryException. NOTE: I can't just pick all the news and mallocs and replace it with customized allocator, it's a lot of work there.

I tried setting Project Properties -> Configuration Properties -> Linker -> System -> Heap Reserve\Commit Size to 256MB or 256000000 but nothing works.

Answer 1

Yes, use the Debug Heap hooks in the CRT.

You can hook malloc to breakpoint when you allocate a large block, using _CrtSetAllocHook and _CrtDbgBreak. Or if your problem is lots of small blocks, you can set a breakpoint on the 10,000th allocation (for example) using _CrtSetBreakAlloc.

• CRT Debug Heap: http://msdn.microsoft.com/en-us/library/974tc9t1%28v=VS.100%29.aspx
• _CrtSetAllocHook: http://msdn.microsoft.com/en-us/library/820k4tb8(v=vs.100).aspx
• _CrtDbgBreak: http://msdn.microsoft.com/en-us/library/k4wx2tde(v=vs.100).aspx
• _CrtSetBreakAlloc: http://msdn.microsoft.com/en-us/library/4wth1ha5(v=vs.100).aspx