Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I quickly read bytes from a memory mapped file in .NET?

Tags:

In some situations the MemoryMappedViewAccessor class just doesn't cut it for reading bytes efficiently; the best we get is the generic ReadArray<byte> which it the route for all structs and involves several unnecessary steps when you just need bytes.

It's possible to use a MemoryMappedViewStream, but because it's based on a Stream you need to seek to the correct position first, and then the read operation itself has many more unnecessary steps.

Is there a quick, high-performance way to read an array of bytes from a memory-mapped file in .NET, given that it should just be a particular area of the address space to read from?

like image 328
Kieren Johnstone Avatar asked Oct 31 '11 15:10

Kieren Johnstone


People also ask

Are memory mapped files faster?

Faster File AccessAccessing files via memory map is faster than using I/O functions such as fread and fwrite . Data are read and written using the virtual memory capabilities that are built in to the operating system rather than having to allocate, copy into, and then deallocate data buffers owned by the process.

How do I use a memory-mapped file?

To work with a memory-mapped file, you must create a view of the entire memory-mapped file or a part of it. You can also create multiple views to the same part of the memory-mapped file, thereby creating concurrent memory. For two views to remain concurrent, they have to be created from the same memory-mapped file.

Why is memory-mapped IO faster?

Accessing memory mapped files is faster than using direct read and write operations for two reasons. Firstly, a system call is orders of magnitude slower than a simple change to a program's local memory.

How big are memory mapped files?

Memory-mapped files cannot be larger than 2GB on 32-bit systems. When a memmap causes a file to be created or extended beyond its current size in the filesystem, the contents of the new part are unspecified.


1 Answers

This solution requires unsafe code (compile with /unsafe switch), but grabs a pointer to the memory directly; then Marshal.Copy can be used. This is much, much faster than the methods provided by the .NET framework.

    // assumes part of a class where _view is a MemoryMappedViewAccessor object      public unsafe byte[] ReadBytes(int offset, int num)     {         byte[] arr = new byte[num];         byte *ptr = (byte*)0;         this._view.SafeMemoryMappedViewHandle.AcquirePointer(ref ptr);         Marshal.Copy(IntPtr.Add(new IntPtr(ptr), offset), arr, 0, num);         this._view.SafeMemoryMappedViewHandle.ReleasePointer();         return arr;     }      public unsafe void WriteBytes(int offset, byte[] data)     {         byte* ptr = (byte*)0;         this._view.SafeMemoryMappedViewHandle.AcquirePointer(ref ptr);         Marshal.Copy(data, 0, IntPtr.Add(new IntPtr(ptr), offset), data.Length);         this._view.SafeMemoryMappedViewHandle.ReleasePointer();     } 
like image 99
Kieren Johnstone Avatar answered Sep 21 '22 16:09

Kieren Johnstone