Read all contents of memory mapped file or Memory Mapped View Accessor without knowing the size of it

Question

I need something similar to ReadToEnd or ReadAllBytes to read all of the contents of the MemoryMappedFile using the MappedViewAccessor if I don't know the size of it, how can I do it?

I have searched for it, I have seen this question, but it is not the thing I am looking for:

How can I quickly read bytes from a memory mapped file in .NET?

Edit:

There is a problem, the (int)stream.Length is not giving me the correct length, it rather gives the size of the internal buffer used! I need to refresh this question because it is very pressing.

Answer 1

Rather use the Stream:

public static Byte[] ReadMMFAllBytes(string fileName)
{
    using (var mmf = MemoryMappedFile.OpenExisting(fileName))
    {
        using (var stream = mmf.CreateViewStream())
        {
            using (BinaryReader binReader = new BinaryReader(stream))
            {
                return binReader.ReadBytes((int)stream.Length);
            }
        }
    }
}

Answer 2

This is difficult to answer since there are still many details of your application that you haven't specified, but I think both Guffa's and Amer's answers are still partially correct:

• A MemoryMappedFile is more memory than file; it is a sequence of 4Kb pages in memory. So, stream.Length will in fact give you all of the bytes (there is no "internal buffer size"), but it might give you more bytes than you expect since the size will always be rounded up to a 4Kb boundary.
• The "file" semantic comes from associating the MemoryMappedFile to a real filesystem file. Assuming that the process which creates the file always adjusts the file size, then you can get the precise size of the file via the fileSystem.

If all of the above would fit your application, then the following should work:

    static byte[] ReadMemoryMappedFile(string fileName)
    {
        long length = new FileInfo(fileName).Length;
        using (var stream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.ReadWrite))
        {
            using (var mmf = MemoryMappedFile.CreateFromFile(stream, null, length, MemoryMappedFileAccess.Read, null, HandleInheritability.Inheritable, false))
            {
                using (var viewStream = mmf.CreateViewStream(0, length, MemoryMappedFileAccess.Read))
                {
                    using (BinaryReader binReader = new BinaryReader(viewStream))
                    {
                        var result = binReader.ReadBytes((int)length);
                        return result;
                    }
                }
            }
        }
    }

To write the data, you can use this:

    private static void WriteData(string fileName, byte[] data)
    {
        using (var stream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite))
        {
            using (var mmf = MemoryMappedFile.CreateFromFile(stream, null, data.Length, MemoryMappedFileAccess.ReadWrite, null, HandleInheritability.Inheritable, true))
            {
                using (var view = mmf.CreateViewAccessor())
                {
                    view.WriteArray(0, data, 0, data.Length);
                }
            }

            stream.SetLength(data.Length);  // Make sure the file is the correct length, in case the data got smaller.
        }
    }

But, by the time you do all of the above you might do just as well to use the file directly and avoid the memory mapping. If mapping it to the filesystem isn't acceptable, then Guffa's answer of encoding the length (or an end marker) in the data itself is probably best.

Answer 3

You can't do that.

A view accessor is created with a minimum size of a system page, which means that it may be larger than the actual file. A view stream is just a stream form of an accessor, so it will also give the same behaviour.

"views are provided in units of system pages, and the size of the view is rounded up to the next system page size"

http://msdn.microsoft.com/en-us/library/dd267577.aspx

The accessor will gladly read and write outside the actual file without throwing an exception. When reading, any bytes outside the file will just be zero. When writing, the bytes written outside the file are just ignored.

To read the file from a memory mapped file with the exact size of the original file, you have to already know that size.

Answer 4

Stream created by MemoryMappedFile has a length aligned to file system page size (usually 4096). You have to get the file size from somewhere else. If it is memory mapped file you could use that code:

byte[] ReadAllMemoryMappedFileBytes(string filePath)
{
    var fileInfo = new FileInfo(filePath);
    using (var file = MemoryMappedFile.CreateFromFile(filePath, FileMode.Open))
    using (var stream = file.CreateViewAccessor())
    {
        byte[] bytes = new byte[fileInfo.Length];
        stream.ReadArray(0, bytes, 0, bytes.Length);
        return bytes;
    }
}