Memory management for intentionally memory intensive applications

Question

Note: I am aware of the question Memory management in memory intensive application, however that question appears to be about applications that make frequent memory allocations, whereas my question is about applications intentionally designed to consume as much physical memory as is safe.

I have a server application that uses large amounts of memory in order to perform caching and other optimisations (think SQL Server). The application runs on a dedicated machine, and so can (and should) consume as much memory as it wants / is able to in order to speed up and increase throughput and response times without worry of impacting other applications on the system.

The trouble is that if memory usage is underestimated, or if load increases its possible to end up with nasty failures as memory allocations fail - in this situation obviously the best thing to do is to free up memory in order to prevent the failure at the expense of performance.

Some assumptions:

• The application is running on a dedicated machine
• The memory requirements of the application exceed the physical memory on the machine (that is, if additional memory was available to the application it would always be able to use that memory to in some way improve response times or throughput)
• The memory is effectively managed in a way such that memory fragmentation is not an issue.
• The application knows what memory can be safely freed, and what memory should be freed first for the least performance impact.
• The app runs on a Windows machine

My question is - how should I handle memory allocations in such an application? In particular:

• How can I predict whether or not a memory allocation will fail?
• Should I leave a certain amount of memory free in order to ensure that core OS operations remain responsive (and don't in that way adversely impact the applications performance), and how can I find out how much memory that is?

The core objective is to prevent failures as a result of using too much memory, while at the same time using up as much memory as possible.

I'm a C# developer, however my hope is that the basic concepts for any such app are the same regardless of the language.

Answer 1

In linux, the memory usage percentage is divided into following levels.

0 - 30% - no swapping 30 - 60% - swap dirty pages only 60 - 90% - swap clean pages also based on LRU policy.

90% - Invoke OOM(Out of memory) killer and kill the process consuming maximum memory.

check this - http://linux-mm.org/OOM_Killer

In think windows might have similar policy, so you can check the memory stats and make sure you never get to the max threshold.

One way to stop consuming more memory is to go to sleep and give more time for memory cleanup threads to run.