Maximum Memory a .NET process can allocate

Question

What is the maximum memory the garbage collector can allocate for a .NET process? When i compile to x64, Process.GetCurrentProcess.MaxWorkingSet returns about 1,4GB, but when i compile to AnyCPU (x64) the same number is returned. For x64 it should be more like the "Limit" value that is displayed in the Task Manager. How can i get the correct number that will cause OutOfMemory-Exceptions when exceeded in all cases?

Some examples what the method should return:

1) Machine Configuration: x64-Windows, 4GB physical memory, 4GB page file
-As 64-Bit process: 8GB
-As 32-Bit process: 1.4GB

2) Machine Configuration: x64-Windows, 1GB physical memory, 2GB page file
-As 64-Bit process: 3GB
-As 32-Bit process: 1.4GB

3) Machine Configuration: x32-Windows, 4GB physical memory, 4GB page file
-As 64-Bit process: Won't happen
-As 32-Bit process: 1.4GB

4) Machine Configuration: x32-Windows, 512MB physical memory, 512MB page file
-As 64-Bit process: Won't happen
-As 32-Bit process: 1.0GB

Answer 1

Windows can be configured to allocate more page file space on demand, or on request.
Job objects can prevent the consumption of more than a certain amount of memory.
Fragmentation of the heap and the generational nature of it (plus the need to put large stuff in the Large Object Heap)

All these mean that the hard limit is not much use in reality and means answering the question "how much memory could I theoretically allocate" is rather more complex than you think.

Since it is complex anyone asking that question is probably trying to do something wrong and should redirect their question to something more useful.

What are you trying to do that would appear to necessitate such a question?

"I just want to know when the current memory load of the process could get problematic so I can take actions like freeing some items of a custom cache."

Right. this is much more tractable a question.

Two solutions in order of complexity:

1. Make your caches use WeakReferences
   • This means that things will be freed by the system almost magically for you but you will have little control over things like the replacement policy
   • this relies on the cached data being much bigger than the key and the overhead of a weak reference
2. Register for notification of Garbage Collections
   • This lets you take control of freeing things up.
   • you rely on the system having the appropriate maximum size for the GC generations which may take a while to get to a steady state.

Points to note. Is it really less expensive to maintain this massive cache (going to disk by the sounds of it) than to recalculate/re-request the data.
If your cache exhibits poor locality between commonly/consecutively requested items then much effort will be spent paging data in and out. A smaller cache with an effective tuned relpacement policy stands a good chance of performing considerably better (and with much less impact on other running programs)

As an aside: In .Net, no variable sized object (strings, arrays) can be more than 2GB in size due to limitations of the core CLR structures for memory management. (and either solution above will benefit from this)