Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How big can a memory-mapped file be?

What limits the size of a memory-mapped file? I know it can't be bigger than the largest continuous chunk of unallocated address space, and that there should be enough free disk space. But are there other limits?

like image 695
user88185 Avatar asked Apr 07 '09 16:04

user88185


People also ask

Are memory-mapped files faster?

Faster File AccessAccessing files via memory map is faster than using I/O functions such as fread and fwrite . Data are read and written using the virtual memory capabilities that are built in to the operating system rather than having to allocate, copy into, and then deallocate data buffers owned by the process.

What is the purpose of memory-mapped file?

A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory.

Why is memory-mapped IO faster?

Accessing memory mapped files is faster than using direct read and write operations for two reasons. Firstly, a system call is orders of magnitude slower than a simple change to a program's local memory.

Are memory-mapped files thread safe?

Yes. If one thread changes part of the data in the mapping, then all other threads immediately see that change. You need to ensure the threads coordinate their changes so no thread is accessing an inconsistent view (eg.


1 Answers

You're being too conservative: A memory-mapped file can be larger than the address space. The view of the memory-mapped file is limited by OS memory constraints, but that's only the part of the file you're looking at at one time. (And I guess technically you could map multiple views of discontinuous parts of the file at once, so aside from overhead and page length constraints, it's only the total # of bytes you're looking at that poses a limit. You could look at bytes [0 to 1024] and bytes [240 to 240 + 1024] with two separate views.)

In MS Windows, look at the MapViewOfFile function. It effectively takes a 64-bit file offset and a 32-bit length.

like image 160
Jason S Avatar answered Sep 30 '22 16:09

Jason S