Platform independent memory mapped [file] IO

Question

I've spent some time investigating memory mapped IO for an application I'm working on. I have some very large (TB scale) files, and I want to map segments from them into memory, for both reading and writing, making maximum use of OS-level caching. The software I'm writing needs to work under Unix/Linux and Windows... performance is critical.

I've discovered boost::iostreams::mapped_file_source and boost::iostreams::mapped_file_sink, which provide most of the facilities I'm looking for. The facilities I'd like, but haven't found are:

• Forcing a synchronisation of written data to disk ( msync(2) on Unix; FlushViewOfFile on Windows)
• Locking of files to prevent two processes attempting to write the same file at the same time (or read while the file is still being written..)
• Controlling attributes of the file at creation time (Unix)

Can I do these things using "boost/iostreams/device/mapped_file.hpp"? Are there other platform independent libraries that would better suit my requirements? Must I develop my own cross-platform library to get this flexibility?

Answer 1

Look at boost::interprocess and boost::interprocess::file_mapping. They have everything you need.

boost::interprocess

boost::interprocess::file_mapping