Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does the memory mapped file ever need to be flushed when access is RDWR?

I was reading through one of golang's implementation of memory mapped files, https://github.com/edsrzf/mmap-go/. First he describes the several access modes:

// RDONLY maps the memory read-only.
// Attempts to write to the MMap object will result in undefined behavior.
RDONLY = 0
// RDWR maps the memory as read-write. Writes to the MMap object will update the
// underlying file.
RDWR = 1 << iota
// COPY maps the memory as copy-on-write. Writes to the MMap object will affect
// memory, but the underlying file will remain unchanged.
COPY

But in gommap test file I see this:

func TestReadWrite(t *testing.T) {
  mmap, err := Map(f, RDWR, 0)
  ... omitted for brevity...
  mmap[9] = 'X'
  mmap.Flush()

So why does he need to call Flush to make sure the contents are written to the file if the access mode is RDWR?

Or is the OS managing this so it only writes when it thinks it should?

If the last option, could you please explain it in a little more detail - what i read is that when the OS is low in memory it writes to the file and frees up memory. Is this correct and does it apply only to RDWR or only to COPY?

Thanks

like image 764
scc Avatar asked Nov 18 '13 19:11

scc


People also ask

What is memory-mapped file access?

A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory.

When you write to a memory-mapped file when is the file actually written to the disk?

4 Answers. Save this answer. Show activity on this post. A memory mapped file is actually partially or wholly mapped in memory (RAM), whereas a file you write to would be written to memory and then flushed to disk.

Are memory mapped files thread safe?

Yes. If one thread changes part of the data in the mapping, then all other threads immediately see that change.

What does memory mapping do?

Memory-mapping is a mechanism that maps a portion of a file, or an entire file, on disk to a range of addresses within an application's address space. The application can then access files on disk in the same way it accesses dynamic memory.


1 Answers

The program maps a region of memory using mmap. It then modifies the mapped region. The system isn't required to write those modifications back to the underlying file immediately, so a read call on that file (in ioutil.ReadAll) could return the prior contents of the file.

The system will write the changes to the file at some point after you make the changes. It is allowed to write the changes to the file any time after the changes are made, but by default makes no guarantees about when it writes those changes. All you know is that (unless the system crashes), the changes will be written at some point in the future.

If you need to guarantee that the changes have been written to the file at some point in time, then you must call msync.

The mmap.Flush function calls msync with the MS_SYNC flag. When that system call returns, the system has written the modifications to the underlying file, so that any subsequent call to read will read the modified file.

The COPY option sets the mapping to MAP_PRIVATE, so your changes will never be written back to the file, even if you using msync (via the Flush function).

Read the POSIX documentation about mmap and msync for full details.

like image 99
rob mayoff Avatar answered Sep 21 '22 08:09

rob mayoff