Design pattern for try/catch block for OutOfMemoryException in .NET

Question

I have an application that works with large amounts of data, and I'm thinking that, may be, sometimes the OutOfMemoryException will be thrown (For half a year, I got no single exception, but I'm just want to know all about it). As I've investigated, after this exception I can't proceed with execution of my program.

Is there any good pattern to handle such exceptions, especially to working with IDisposable classes?

Answer 1

In a genuine OOM scenario (more likely on x86 than x64) you are pretty doomed there. Almost anything would cause an allocation, so your best option is to die as quickly and elegantly as possible, doing minimum harm.

Since it isn't happening, don't stress overly, but avoidance is better than handling here:

• use streaming data APIs rather than buffering everything in memory
• re-use buffers etc
• avoid enormous arrays/lists/etc (in truth, the most likely way to cause an OOM is to request an enormous (but single) array) - for example, a jagged array scales better than a 2D array (even on x64 there is a hard limit on the maximum size of a single array)
• think about how you handle sparse data
• do you read lots of strings from external sources? If so, consider around a custom interner so you don't have 20,000 different copies of common strings (country names, for example)
• keep an eye on what you release when
• avoid accidentally prolonged life on objects, especially via event subscriptions (notorious for accidental extensions to lifetimes)

Answer 2

There is no good pattern, OOM is a nasty exception that can strike at any moment. Which makes it almost an asynchronous exception. The only odds you have for handling it is when your code is structured to allocate large amounts of memory at the same time. So you'll have some odds to back out and unwind the program state as though nothing happened.

You need to design your program so it never needs to allocate more than about half of all available virtual memory, one gigabyte on a 32-bit machine. Dragons live beyond that amount, your program will fail on an allocation of 90 MB or less, even if there is another 500 MB of virtual memory unused. A problem induced by address space fragmentation. If you routinely cross this threshold then you need to switch to a 64-bit operating system.

Answer 3

The two answers before mine are correct and sound advice,
but there's one thing they haven't mentioned - which is load testing.
If you're concerned that under certain load your application might run out of memory- test it against that load (and preferably- larger loads).

In my previous job I used such a tool (HP's Performance Center) and it proved invaluable not only to check for errors and limits of our system, but also in identifying bottlenecks and costly operations.