How to implement platform independent asynchronous write to file?

Question

I am creating a program that will receive messages from a remote machine and needs to write the messages to a file on disk. The difficulty I am finding lies in the fact that the aim of this program is to test the performance of the library that receives the messages and, therefore, I need to make sure that writing the messages to disk does not affect the performance of the library. The library delivers the messages to the program via a callback function. One other difficulty is that the solution must be platform independent.

What options do I have?

I thought of the following:

• using boost:asio to write to file, but it seems (see this documentation) that asynchronous write to file is in the Windows specific part of this library - so this cannot be used.
• Using boost::interprocess to create a message queue but this documentation indicates that there are 3 methods in which the messages can be sent, and all methods would require the program to block (implicitly or not) if the message queue is full, which I cannot risk.
• creating a std::deque<MESSAGES> to push to the deque from the callback function, and pop the messages out while writing to the file (on a separate thread), but STL containers are not guaranteed to be thread-safe. I could lock the pushing onto, and popping off, the deque but we are talking about 47 microseconds between successive messages so I would like to avoid locks altogether.

Does anyone have any more ideas on possible solutions?

Answer 1

STL containers may not be thread-safe but I haven't ever hit one that cannot be used at different times on different threads. Passing the ownership to another thread seems safe.

I have used the following a couple of times, so I know it works:

• Create a pointer to a std::vector.
• Create a mutex lock to protect the vector pointer.
• Use new[] to create a std::vector and then reserve() a large size for it.

In the receiver thread:

• Lock the mutex whenever adding an item to the queue. This should be a short lock.
• Add queue item.
• Release the lock.
• If you feel like it signal a condition variable. I sometimes don't: it depends on the design. If the volume is very high and there is no pause in the receive side just skip the condition and poll instead.

On the consumer thread (the disk writer):

• Go look for work to do by polling or waiting on a condition variable:
• Lock the queue mutex.
• Look at the queue length.
• If there is work in the queue assign the pointer to a variable in the consumer thread.
• Use new[] and reserve() to create a new queue vector and assign it to the queue pointer.
• Unlock the mutex.
• Go off and write your items to disk.
• delete[] the used-up queue vector.

Now, depending on your problem you may end up needing a way to block. For example in one of my programs if the queue length ever hits 100,000 items, the producing thread just starts doing 1 second sleeps and complaining a lot. It is one of those things that shouldn't happen, yet does, so you should consider it. Without any limits at all it will just use all the memory on the machine and then crash with an exception, get killed by OOM or just come to a halt in a swap storm.

Answer 2

boost::thread is platform independent so you should be able to utilize that create a thread to do the blocking writes on. To avoid needing to lock the container every time a message is placed into the main thread you can utilize a modification on the double buffering technique by creating nested containers, such as:

std::deque<std::deque<MESSAGES> >

Then only lock the top level deque when a deque full of messages is ready to be added. The writing thread would in turn only lock the top level deque to pop off a deque full of messages to be written.