Why malloc always return NULL

Question

My dev environment is VS2008, DX9, Windows XP. I try to add protection handling to the out of memory case. When malloc return NULL, I would page some resource to disk, and release the resources in the memory.

But sometimes, malloc always return NULL, even if I release most of resources and process memory usage and VM size is only 800MB in task manager.

I think to use malloc to allocate 88 bytes should be fine, where process memory usage is only 800MB. But malloc always return NULL.

Could this be memory fragmentation? It doesn't look like that, since process memory usage is not too much.

alt text http://i.imagehost.org/0267/Snap2.jpg

Answer 1

You mentioned memory fragmentation and that would certainly be my first guess. Try downloading this application. Its called Address Space Monitor, and should be able to show you if its a fragmentation issue.

Answer 2

It could be virtual address space fragmentation. One way to check is to call HeapCompact(GetProcessHeap(), 0). If that frees up enough memory, then that's the likely cause.

Another similar cause would be launching from the debugger; that causes Windows to use the debug heap, which has really really bad memory behavior over a long period of time. To disable that behavior, set _NO_DEBUG_HEAP=1 in the environment and run.