What, exactly, do MemoryCache's memory limits mean?

Question

System.Runtime.Caching.MemoryCache is a class in the .NET Framework (version 4+) that caches objects in-memory, using strings as keys. More than System.Collections.Generic.Dictionary<string, object>, this class has all kinds of bells and whistles that let you configure how much big the cache can grow to (in either absolute or relative terms), set different expiration policies for different cache items, and so much more.

My questions relate to the memory limits. None of the docs on MSDN seem to explain this satisfactorily, and the code on Reference Source is fairly opaque. Sorry about piling all of this into one SO "question", but I can't figure out how to take some out into their own questions, because they're really just different views of one overall question: "how do you reconcile idiomatic C#/.NET with the notion of a generally useful in-memory cache that has configurable memory limits that's implemented nearly entirely in managed code?"

1. Do key sizes count towards the space that the MemoryCache is considered to take up? What about keys in the intern pool, each of which should only add the size of an object reference to the size of the cache?
2. Does MemoryCache consider more than just the size of the object references that it stores when determining the size of the object being stored in the pool? I mean... it has to, right? Otherwise, the configuration options are extremely misleading for the common-case... for the remaining questions, I'm going to assume that it does.
3. Given that MemoryCache almost certainly considers more than the size of the object references of the values stored in the cache, how deep does it go?
   1. If I were implementing something like this, I would find it very difficult to consider the memory usage of the "child" members of individual objects, without also pulling in "parent" reference properties.
   2. e.g., imagine a class in a game application, Player. Player has some player-specific state that's encapsulated in a public PlayerStateData PlayerState { get; } property that encapsulates what direction the player is looking, how many sprockets they're holding, etc., as well as a reference to the entire game's state public GameStateData GameState { get; } that can be used to get back to the game's (much larger) state from a method that only knows about a player.
   3. Does MemoryCache consider both PlayerState and GameState when considering the size of the contribution to the cache?
   4. Maybe it's more like "what's the total size on the managed heap taken up by the objects directly stored in the cache, and everything that's reachable through members of those objects"?
   5. It seems like it would be silly to multiply the size of GameState's contribution to the limit by 5 just because 5 players are cached... but then again, a likely implementation might do just that, and it's difficult to count PlayerState without counting GameState.
4. If an object is stored multiple times in the MemoryCache, does each entry count separately towards the limit?
5. Related to the previous one, if an object is stored directly in the MemoryCache, but also indirectly through another object's members, what impact does either one have on the memory limit?
6. If an object is stored in the MemoryCache, but also referenced by some other live objects completely disconnected from the MemoryCache, which objects count against the memory limit? What about if it's an array of objects, some (but not all) of which have incoming external references?

My own research led me to SRef.cs, which I gave up on trying to understand after getting here, which later leads here. Guessing the answers to all these questions would revolve around finding and meditating on the code that ultimately populated the INT64 that's stored in that handle.

Answer 1

I know this is late but I've done a lot of digging in the source code to try to understand what is going on and I have a fairly good idea now. I will say that MemoryCache is the worst documented class on MSDN, which kind of baffles me for something intended to be used by people trying to optimize their applications.

MemoryCache uses a special "sized reference" to measure the size of objects. It all looks like a giant hack in the memory cache source code involving reflection to wrap an internal type called "System.SizedReference", which from what I can tell causes the GC to set the size of the object graph it points to during gen 2 collections.

From my testing, this WILL include the size of parent objects, and thus all child objects referenced by the parent etc, BUT I've found that if you make references to parent objects weak references (i.e. via WeakReference or WeakReference<>) then it is no longer counted as part of the object graph, so that is what I do for all cache objects now.

I believe cache objects need to be completely self-contained or use weak references to other objects for the memory limit to work at all.

If you want to play with it yourself, just copy the code from SRef.cs, create an object graph and point a new SRef instance to it, and then call GC.Collect. After the collection the approximate size will be set to the size of the object graph.