.NET MemoryCache: How does it enforce memory limit?

Question

.NET MemoryCache is a cache of C# objects. Some objects can have a complex structure, and other can have unsafe references. Is C# doing some magic for implementing the PhysicalMemoryLimit or is it just computing the shallow size of each object?

I suspect the later is the case. Still, if I put the same object multiple times in the cache (for tracking missing items, for instance), will the size be accounted a single time, or for each entry that contains that instance?

Answer 1

The .NET MemoryCache is similar to the ASP.NET Cache class. If we look at the ASP.NET Cache we see a function called CacheItemRemovedCallback. This is triggerd when a Item is removed from the Cache.

This function gives a CacheItemRemovedReason with the callback function. If we look at the reasons, we see that a item can be removed from the cache because the system removed it to free memory. So while the PhysicalMemoryLimit gives the percentage of physical memory that the cache can use in a single tread, I think they leave it over to the system to clear the cache if it reach the limit.

If you really put a Cache item into the cache with the Add function it will add it as an new CacheItem instance. So it will be accounted multiple times. If you use the function AddOrGetExisting it will check if the item is allready in the cache. If so it will use that instance and not a new instance. So then it will be accounted once.

Hope this helps you in the right direction.

Answer 2

Reading the documentation, it appears the cache makes no attempt to compute the size of the objects it is caching. This makes sense because it is not something that can be done from within a process itself for arbitrary types (you can do it for fixed size structs, or arrays of fixed size structs, but that is about it); a bit of googling will confirm that to you. It does however know how much RAM is available on the computer; you can get this yourself from new Microsoft.VisualBasic.Devices.ComputerInfo().AvailablePhysicalMemory. So presumably the cache does two things:

1. It tracks when each object was last used.
2. Polls for memory statistics at some interval.

Then on each poll either the amount of available memory is within acceptable limits, or it is not. If it is within acceptable limits it does nothing. If it is not it starts removing items, with the item that was last accessed longest ago removed first. It keeps removing items until the memory gets back within acceptable limits.

If you think about it that is pretty much all you can do with the information available to the cache.

This strategy is OK, but it obviously breaks down if you have other objects holding references to items in the cache, because removing the item from the cache will not free it up for garbage collection. That is the point of the callback, to perform the clean-up to ensure there are no more references to the object.

Answer 3

Here's the source. The answer to your second question is obvious if you look at the implementation on Add method: referencesource.microsoft.com that is calling AddOrGetExisting.

I don't know about the size, but I guess that you're right in your assumption that there is no magic at all. Also if you're interested you may inspect the sources in-depth.