istream vs memory mapping a file?

Question

I am trying to map a file to memory and then parse line by line- is istream what I should be using?

Is istream the same as mapping a file to memory on Windows? I have had difficulties trying to find a complete example of mapping a file into memory.

I have seen people link memory mapping articles from MSDN, but if anybody could recommend a small (~15 line?) example I would be most thankful.

I must be searching for the wrong thing, but when searching "C++ memory mapping example" on Google, I could not find an example that included iterating through.

These were the closest results (just so people realize I have looked):

• http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2044.html#ClassSharedMemoryObjectExample
• http://msdn.microsoft.com/en-us/library/dd997372.aspx (no C++ code)
• https://beej.us/guide/bgipc/html/multi/mmap.html (was for Unix I believe, not Windows)

Answer 1

std::istream is an abstract type – you cannot use it directly. You should be deriving from it with a custom array-backed streambuf:

#include <cstddef>
#include <string>
#include <streambuf>
#include <istream>

template<typename CharT, typename TraitsT = std::char_traits<CharT>>
struct basic_membuf : std::basic_streambuf<CharT, TraitsT> {
    basic_membuf(CharT const* const buf, std::size_t const size) {
        CharT* const p = const_cast<CharT*>(buf);
        this->setg(p, p, p + size);
    }

    //...
};

template<typename CharT, typename TraitsT = std::char_traits<CharT>>
struct basic_imemstream
: virtual basic_membuf<CharT, TraitsT>, std::basic_istream<CharT, TraitsT> {
    basic_imemstream(CharT const* const buf, std::size_t const size)
    : basic_membuf(buf, size),
      std::basic_istream(static_cast<std::basic_streambuf<CharT, TraitsT>*>(this))
    { }

    //...
};

using imemstream = basic_imemstream<char>;

char const* const mmaped_data = /*...*/;
std::size_t const mmap_size = /*...*/;
imemstream s(mmaped_data, mmap_size);
// s now uses the memory mapped data as its underlying buffer.

As for the memory-mapping itself, I recommend using Boost.Interprocess for this purpose:

#include <cstddef>
#include <string>
#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/mapped_region.hpp>

namespace bip = boost::interprocess;

//...

std::string filename = /*...*/;
bip::file_mapping mapping(filename.c_str(), bip::read_only);
bip::mapped_region mapped_rgn(mapping, bip::read_only);
char const* const mmaped_data = static_cast<char*>(mapped_rgn.get_address());
std::size_t const mmap_size = mapped_rgn.get_size();

---

^{Code for imemstream taken from this answer by Dietmar Kühl.}

Answer 2

Is istream the same as mapping a file to memory on windows?

Not exactly. They are not the same in the same sense a "stream" is not a "file".

Think to a file as a stored sequence, and to a stream as the interface for the "channel" (a stream_buffer) that sequence flows when moving from its store towards the receiving variables.

Think to a memory mapped file as a "file" that -instead been stored outside the processing unit- is stored in-sync in memory. It has the advantage to be visible as a raw memory buffer being a file. If you want to read it as a stream, the simplest way is probably using a istringstream that has that raw buffer as the place to read from.