Out of memory on _beginthreadex

Question

I currently debug a multi threaded application, which runs without errors until some functions where called about 2000 times. After that the application stops responding, which I could track down to _beginthreadex failing with an out of memory error.

When examining the Application in ProcessExplorer I can see a growing number of thread handles leaked and a growing virtual memory until the error occurs, the private bytes stay low. The leaked threads also call CoInitialize and never call CoUninitialize.

What I would like to know is:

• What does the Virtual memory represent ?
• Is the virtual memory related to the leaked thread handles?
• Does COM or MSXML6 (called by the threads) copy thread handles and how can I Close them?

I hope that my question is clear and doesn't break any roules,it is my first question and english isn't my first language.:-(

I forgot to mention, I close the handles returned by _beginthreadex once the threads get terminated, which reduces the number of open handles by about half but does not affect the virtual memory. Additionally before i inserted the CloseHandle call each thread handle shown in ProcessExplorer had a handle count of two for the thread.

Edit

I fell stupid for not including this before, I know that the threads exit as the number of active threads while debugging with visual studio does not grow. And I do hope that not all of the leaked memory is a result of calls to TerminateThread as they are used in a rather big library and I would prefer not modifying that.

To the com part of my question, with !htrace -diff i find thread handles allocated by msxml but not freed after the functioncalls end, could they be related to the leak or will they be Closed at a later time?

Thanks for all those comments, while the problem is still there they helped me understand it better.

Answer 1

The virtual memory available to a process is 2Gb of the 4Gb address space. Each Thread reserves about 1Mb of virtual memory space by default for its stack space. win32 applications therefore have a limit of about 2000 live threads before virtual memory becomes exhausted.

Virtual memory is the memory that applications get in modern virtual memory OS's like windows. What happens on Win32 is, your application gets given a 2Gb virtual address space. When your program calls new or malloc, after tunneling trhough several layers, space gets allocated for your application ON DISK - in the pagefile. When CPU instructions try to access that memory, hardware exceptions get thrown and the kernel allocates physical RAM to that area, and reads the contents from the pagefile. So, regardless of the physical RAM in the PC, each and every application believes it has access to a whole 2Gb. Virtual Memory is a count of how much of your 2Gb space has been used up.

Each thread (see above) reserves 1 Mb of virtual address space for its stack to grow. Most of that 1Mb is just reserved space (hopefully) without the backing of RAM or pagefile.

When you close a thread handle you do NOT close the thread. threads are terminated by another thread calling TerminateThread (which leaks the threads stack and some other resources so NEVER use it), calling ExitThread() themselves, or by exiting from their ThreadProc.

So, with the 2000 call limit, the unmatched CoInitialize and CoUninitialize calls, I would say that your threads are not exiting cleanly or at all. Each of the 2000 worker threads is stuck doing something rather than exiting after finishing their work.

Answer 2

The _beginthreadex/_endthreadex functions do not automatically close the thread handle, so you must call the win32 CloseHandle function to close it. The handle is the value returned by _beginthreadex. If you use _beginthread/_endthread instead, the handle will be closed automatically.

Regarding virtual memory: It represents memory that was reserved, but is not necessarily used yet. The leaked memory (or at least part of it) is related to the handle leak. When the last handle to a thread is closed, Windows will release the virtual memory reserved for it.