How to make a .NET process run out of memory without exhausting all system memory

Question

The problem is simple I have a process, that does ETL on some xml files. We started getting really large xml files and I started getting OutOfMemoryExceptions.

Fixing the process is relatively simple. However, I'd like to make a unit test for my NUnit suite to make sure the process will continue to be able to handle really large files. However, actually running out of memory on my development workstation slows down my machine, and is time consuming. Storing a humongous test file in version control is also a bad idea. If I could artificially limit a process, thread or appdomain to only use a fixed amount of ram, lets say 128 megs, I could make a smaller unit test that would not bring my workstation, to its knees.

Any suggestions? Is their some unmanaged API I can P/Invoke?

Answer 1

Can't you use a mocking framework for the memory allocation and have it throw OutOfMemoryException as one of the tests?

Having said that though, if you really have run out of memory there's not much your application can safely do, but if you can at least fail gracefully your users will be grateful.

An example: I had a case in a previous job where we were displaying 3D models of factories in real-time. The models grew so large that when we tried to load textures we'd get out of memory failures. We managed to keep the application alive and rendering by making sure that the code coped with null pointers even though the rest of the code thought that there should be texture information there.

Answer 2

Mocking is best. Actually raising an OOM is by defininition not a unit test. When dealing with memory, you are dealing with load testing. If you read the links at the bottom of this email, you'll find real OOMs are dastardly hard to reproduce and debug in the best of cases. A contrived OOM exception is not the true cause of the exception, and thus no more interesting than a mock for testing.

Stick with a unit test using a mock for validation. If you still get OOMs, throw more memory on your server and make your process recycle/ restart more often.

Here is some interesting reading on OutMemoryExceptions I collected the last time I fought with them. Summary: OOMs occur when the system can't allocate the amount that you requested - which doesn't mean you are out of memory.

• MSDN ManagedLeaks
• CLR Inside Out
• MSDN Debuging
• Bugslayer column on AD Plus