Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory-Mapped File is Faster on Huge Sequential Read? Why?

I used the code below to measure the performance difference between reading large, sequential reads of a memory-mapped file, as compared to just calling ReadFile:

HANDLE hFile = CreateFile(_T("D:\\LARGE_ENOUGH_FILE"),
    FILE_READ_DATA, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING,
    FILE_FLAG_NO_BUFFERING, NULL);
__try
{
    const size_t TO_READ = 32 * 1024 * 1024;
    char sum = 0;
#if TEST_READ_FILE
    DWORD start = GetTickCount();
    char* p = (char*)malloc(TO_READ);
    DWORD nw;
    ReadFile(hFile, p, TO_READ, &nw, NULL);
#else
    HANDLE hMapping = CreateFileMapping(hFile, NULL, PAGE_READONLY,
        0, 0, NULL);
    const char* const p = (const char*)MapViewOfFile(hMapping,
        FILE_MAP_READ, 0, 0, 0);
    DWORD start = GetTickCount();
#endif
    for (size_t i = 0; i < TO_READ; i++)
    {
        sum += p[i]; // Do something kind of trivial...
    }
    DWORD end = GetTickCount();
    _tprintf(_T("Elapsed: %u"), end - start);
}
__finally { CloseHandle(hFile); }

(I just changed the value of TEST_READ_FILE to change the test.)

To my surprise, ReadFile was slower by ~20%! Why?

like image 536
user541686 Avatar asked Mar 10 '11 08:03

user541686


People also ask

Why are memory mapped files faster?

Accessing files via memory map is faster than using I/O functions such as fread and fwrite . Data are read and written using the virtual memory capabilities that are built in to the operating system rather than having to allocate, copy into, and then deallocate data buffers owned by the process.

Is writing to a memory-mapped file faster?

Performance: Memory mapped writing is often fast as no stream/file buffers are used. OS does the actual file writing, usually in blocks of several kilo bytes at once. One downside would be, unless you're writing sequentially there could be page faults slowing down your program.

Why is mmap faster than read?

What mmap helps with is that there is no extra user space buffer involved, the "read" takes place there where the OS kernel sees fit and in chunks that can be optimized. This may be an advantage in speed, but first of all this is just an interface that is easier to use.

What are the advantages of memory-mapped?

Benefits. The benefit of memory mapping a file is increasing I/O performance, especially when used on large files. For small files, memory-mapped files can result in a waste of slack space as memory maps are always aligned to the page size, which is mostly 4 KiB.


1 Answers

FILE_FLAG_NO_BUFFERING cripples ReadFile. The memory-mapped file is free to use whatever read-ahead algorithm it wants, and you've forbidden ReadFile to do the same. You've turned off caching only in the ReadFile version. Memory-mapped files can't work without file cache.

like image 146
Ben Voigt Avatar answered Nov 15 '22 12:11

Ben Voigt