Why isn't there hardware support for memory management?

Question

Virtual memory is well-supported by modern hardware, but application-level memory allocation is still all implemented software, be it manual heap memory management a-la C and C++, or VM-level garbage collection.

Going further than the classic manual memory management vs garbage collection debate, why aren't there hardware-based primitives that can help build efficient memory management and/or garbage collection schemes in user-space (possibly exposed by, or built into the OS, and then various VMs).

Hardware is used to accelerate graphics operations, offload various parts of a network stack, and cryptographic algorithms and audio/video codecs are frequently implemented in hardware, why can't building blocks for higher-level memory management be? It seems so ubiquitous, yet I don't know any hardware-assisted implementations.

Given my lack of hardware knowledge it's a bit of a murky area to me, but I'm interested to hear

1. if there is such a thing at all (at least at the research stage), or
2. will or will it not give any benefit over conventional memory management, or alternatively
3. why it is not feasible to build such a thing in the hardware?

Answer 1

You could in theory implement a complete Java VM, including memory management, in hardware, and I believe there are some research projects that (try to) do that. But there are several good reasons not to implement stuff in hardware:

• hardware is fixed, you can't easily patch bugs or implement newer/better algorithms
• hardware is expensive, for complex operations such as garbage collection you'll need a lot of hardware, while a software implementation using existing hardware resources for it is much cheaper
• hardware resources take up space and consume (static) power, even while not in use, while unused software code does relatively little harm

In the end, for each feature you want, you have to make the trade-off between these costs, and the gain you have (faster or lower-powered execution).

For memory management, which are typically complex algorithms but that don't run all that often, the gains will be rather small (you may be able to speed up garbage collection by 10x, but if it only took 1% of execution time to begin with, why bother?) The cost, on the other hand, will be a much bigger chip where much of the area is wasted because it's inactive most of the time...

Answer 2

We had a lot of this hardware stuff in the 70th and 80th of the last millenium. All this Lisp machines were pretty good in trying to help memory management with indirect and double indirect access (required if your GC moves objects around). Some of us also remember the first days of the 80286 where people thought that segments could be used for better memory management and failed terrible on performance.

The current state of wisdom so far is that it is much better to optimize CPU's for general purpose usage instead of adding special features that are required only from time to time.

Modern garbage collectors already use some operating system features like the dirty markings of virtual pages to implement write barriers but other then this the algorithms are pretty simple, straightforward and high level. There isn't really special hardware required.

I just recently found an amazing result when using HP-UX. You can set the virtual page size to 256MB which will effectivly turn of the virtual memory. This gave a 120% performance increase on this CPU. TLB misses are really serious even more then cache misses. This makes me think about the good old MIPS architecture which stored a process id in the TLB so it did not require a complete TLB flush on each process switch.

There is still lot of room for memory management improvements that are more important then some high level Garbage collection features.