I am writing a program in C++ using VS2010 to read a text file and extract certain information from it. I completed the code using filestream and it worked well. However now I am asked to map the file to memory and use it rather than the file operations.
I am absolutely a newbie in case of memory mapping. A part of the code I have written is as follows.
boost::iostreams::mapped_file_source apifile;
apifile.open(LogFileName,LogFileSize);
if(!apifile.is_open())
return FILE_OPEN_ERROR;
// Get pointer to the data.
PBYTE Buffer = (PBYTE)apifile.data();
while(//read till end of the file)
{
// read a line and check if it contains a specific word
}
While using filestream I would have used eof
and getline
and string::find
for performing the operations. But I don't have any idea on how to do it using memory mapped file.
EDIT 1:
int ProcessLogFile(string file_name)
{
LogFileName = file_name;
apifile.open(LogFileName);//boost::iostreams::mapped_file_source apifile(declared globally)
streamReader.open(apifile, std::ios::binary);//boost::iostreams::stream <boost::iostreams::mapped_file_source> streamReader(declared globally)
streamoff Curr_Offset = 0;
string read_line;
int session_id = 0;
int device_id = 0;
while(!streamReader.eof())
{
\\COLLECT OFFSETS OF DIFFERENT SESSIONS
}
streamReader.close();
}
This function worked and i got the offsets to the required structure.
Now after calling this function, I call yet another function as follows:
int GetSystemDetails()
{
streamReader.open(apifile, std::ios::binary);
string read_line;
getline(streamReader,read_line);
cout << "LINE : " << read_line;
streamReader.close();
}
I don't get any data in read_line. Is that memory mapping only for a single function? How can I use the same memory mapped file across different functions?
A memory-mapped file is a feature of the Windows operating system which allows memory to be shared between two or more processes running on the same machine. It requires much less overhead than other methods of inter-process communication such as remoting or WCF.
A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory.
Yes, mmap creates a mapping. It does not normally read the entire content of whatever you have mapped into memory. If you wish to do that you can use the mlock/mlockall system call to force the kernel to read into RAM the content of the mapping, if applicable.
I agree with people questioning the use of a mmap if you just read through the file sequentially.
boost::mapped_file_source
models a Device. There's two approaches to use such a Device:
data()
as you try)You can use the mapped_file_source to report the actual size (m.data()+m.size()
).
Let's take a sample to count lines:
#include <boost/iostreams/device/mapped_file.hpp> // for mmap
#include <algorithm> // for std::find
#include <iostream> // for std::cout
#include <cstring>
int main()
{
boost::iostreams::mapped_file mmap("input.txt", boost::iostreams::mapped_file::readonly);
auto f = mmap.const_data();
auto l = f + mmap.size();
uintmax_t m_numLines = 0;
while (f && f!=l)
if ((f = static_cast<const char*>(memchr(f, '\n', l-f))))
m_numLines++, f++;
std::cout << "m_numLines = " << m_numLines << "\n";
}
You could possibly adapt this. I have several more complicated parsing examples based on memory mapped files:
open()
+read()
was faster than the memory map due the sequential access nature
istream
This gives you all the usual stream-based operations of c++ standard streams, so you can detect the end of the file like you would always:
#include <boost/iostreams/device/mapped_file.hpp> // for mmap
#include <boost/iostreams/stream.hpp> // for stream
#include <algorithm> // for std::find
#include <iostream> // for std::cout
#include <cstring>
int main()
{
using boost::iostreams::mapped_file_source;
using boost::iostreams::stream;
mapped_file_source mmap("test.cpp");
stream<mapped_file_source> is(mmap, std::ios::binary);
std::string line;
uintmax_t m_numLines = 0;
while (std::getline(is, line))
{
m_numLines++;
}
std::cout << "m_numLines = " << m_numLines << "\n";
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With