Using a mutex to block execution from outside the critical section

Question

I'm not sure I got the terminology right but here goes - I have this function that is used by multiple threads to write data (using pseudo code in comments to illustrate what I want)

//these are initiated in the constructor
int* data; 
std::atomic<size_t> size;

void write(int value) {
    //wait here while "read_lock"
    //set "write_lock" to "write_lock" + 1
    auto slot = size.fetch_add(1, std::memory_order_acquire);
    data[slot] = value;
    //set "write_lock" to "write_lock" - 1
}

the order of the writes is not important, all I need here is for each write to go to a unique slot

Every once in a while though, I need one thread to read the data using this function

int* read() {
    //set "read_lock" to true
    //wait here while "write_lock"
    int* ret = data;
    data = new int[capacity];
    size = 0;
    //set "read_lock" to false
    return ret;
}

so it basically swaps out the buffer and returns the old one (I've removed capacity logic to make the snippets shorter)

In theory this should lead to 2 operating scenarios:

1 - just a bunch of threads writing into the container

2 - when some thread executes the read function, all new writers will have to wait, the reader will wait until all existing writes are finished, it will then do the read logic and scenario 1 can continue.

The question part is that I don't know what kind of a barrier to use for the locks -

A spinlock would be wasteful since there are many containers like this and they all need cpu cycles

I don't know how to apply std::mutex since I only want the write function to be in a critical section if the read function is triggered. Wrapping the whole write function in a mutex would cause unnecessary slowdown for operating scenario 1.

So what would be the optimal solution here?

Answer 1

If you have C++14 capability then you can use a std::shared_timed_mutex to separate out readers and writers. In this scenario it seems you need to give your writer threads shared access (allowing other writer threads at the same time) and your reader threads unique access (kicking all other threads out).

So something like this may be what you need:

class MyClass
{
public:
    using mutex_type = std::shared_timed_mutex;
    using shared_lock = std::shared_lock<mutex_type>;
    using unique_lock = std::unique_lock<mutex_type>;

private:
    mutable mutex_type mtx;

public:

    // All updater threads can operate at the same time
    auto lock_for_updates() const
    {
        return shared_lock(mtx);
    }

    // Reader threads need to kick all the updater threads out
    auto lock_for_reading() const
    {
        return unique_lock(mtx);
    }
};

// many threads can call this
void do_writing_work(std::shared_ptr<MyClass> sptr)
{
    auto lock = sptr->lock_for_updates();

    // update the data here
}

// access the data from one thread only
void do_reading_work(std::shared_ptr<MyClass> sptr)
{
    auto lock = sptr->lock_for_reading();

    // read the data here
}

The shared_locks allow other threads to gain a shared_lock at the same time but prevent a unique_lock gaining simultaneous access. When a reader thread tries to gain a unique_lock all shared_locks will be vacated before the unique_lock gets exclusive control.