Is there a BigMemory like system for .net

Question

I have just been reading about how BigMemory allows Java systems to scale up rather than out.

About BigMemory:

BigMemory gives Java applications instant, effortless access to a large memory footprint, free of the constraints of garbage collection.

BigMemory is pure Java and provides an in-process, off-heap cache that lets you store large amounts of data—up to a terabyte—closer to your application.

This breakthrough solution improves memory utilization and application performance with both standalone and distributed caching.

So how do I do the same with .net, e.g. in-process, off-heap cache. (Note the Asp.net cache is on the garbage collected heap)

Answer 1

No, there is not a BigMemory system for .Net (i.e. an in-process non-GC heap memory manager), however, you could roll your own.

You could utilize an unmanaged heap to have a non-garbage collected in-process heap, however, if you are working with objects rather than raw memory, you'll have to serialize and deserialize them which is slow.

You'll need to keep a lookup of heap infos so you can retrieve your objects, this obviously has its own memory overhead, so not suitable for a huge amount of very small objects as:

a. A lot of memory will be taken up by management objects.
b. The GC will go berserk scanning the management objects.

If the objects are large enough and there's not too many of them, this could work for you.

However, you could push some of the management info into the unmanaged heap too. There are lots of optimization opportunities.

This can all be wrapped up to work like a key\value cache, thus abstracting the heap infos and heap.

Updated

Updated sample code to use Protobuf which does binary serialization significantly faster than .Net. This simple sample can Put+Get 425k objects per second, with a key\value wrapper. Your millage will vary depending on object size\complexity.

Object size is stored in unmanaged heap to reduce memory consumption on managed heap.

...
...
using ProtoBuf;

[TestFixture]
public class UnmanagedHeap
{
    [Test]
    public void UnmanagedHeapAccess()
    {
        const int Iterations = 425 * 1000;
        const string Key = "woo";

        Bling obj = new Bling { Id = -666 };
        Cache cache = new Cache();
        Stopwatch sw = Stopwatch.StartNew();

        for (int i = 0; i < Iterations; i++)
        {
            cache.Put(Key, obj);

            obj = cache.Get<Bling>(Key);
        }

        cache.Remove(Key);

        Console.WriteLine(sw.Elapsed.TotalMilliseconds);
    }

    [DataContract]
    public class Bling
    {
        [DataMember(Order = 1)]
        public int Id { get; set; }
    }

    public class Cache
    {
        private const int SizeFieldWidth = 4;

        private readonly Dictionary<string, IntPtr> _lookup = new Dictionary<string, IntPtr>();

        public void Put(string key, object obj)
        {
            IntPtr oldPtr = _lookup.TryGetValue(key, out oldPtr) ? oldPtr : IntPtr.Zero;

            IntPtr newPtr = SerializeToHeap(obj, oldPtr);

            _lookup[key] = newPtr;
        }

        public T Get<T>(string key)
        {
            IntPtr ptr = _lookup[key];

            return DeserializeFromHeap<T>(ptr);
        }

        public void Remove(string key)
        {
            IntPtr ptr = _lookup[key];

            Marshal.FreeHGlobal(ptr);

            _lookup.Remove(key);
        }

        private static IntPtr SerializeToHeap(object obj, IntPtr oldPtr)
        {
            using (MemoryStream ms = new MemoryStream())
            {
                Serializer.Serialize(ms, obj);
                byte[] objBytes = ms.GetBuffer();
                int newSize = (int)ms.Length;
                bool requiresAlloc = true;

                if (oldPtr != IntPtr.Zero)
                {
                    int oldSize = GetObjectSize(oldPtr);

                    requiresAlloc = (oldSize != newSize);
                }

                IntPtr newPtr = requiresAlloc ? Marshal.AllocHGlobal(newSize + SizeFieldWidth) : oldPtr;

                byte[] sizeField = BitConverter.GetBytes(newSize);
                Marshal.Copy(sizeField, 0, newPtr, SizeFieldWidth);
                Marshal.Copy(objBytes, 0, newPtr + SizeFieldWidth, newSize);
                return newPtr;
            }
        }

        private static T DeserializeFromHeap<T>(IntPtr ptr)
        {
            int size = GetObjectSize(ptr);
            byte[] objBytes = new byte[size];
            Marshal.Copy(ptr + SizeFieldWidth, objBytes, 0, size);

            using (MemoryStream ms = new MemoryStream(objBytes))
            {
                return Serializer.Deserialize<T>(ms);
            }
        }

        private static int GetObjectSize(IntPtr ptr)
        {
            byte[] sizeField = new byte[SizeFieldWidth];
            Marshal.Copy(ptr, sizeField, 0, SizeFieldWidth);
            int size = BitConverter.ToInt32(sizeField, 0);
            return size;
        }
    }
}