Atomic access to shared memory

Question

I have a shared memory between multiple processes that interpets the memory in a certain way. Ex:

DataBlock {
int counter;
double value1;
double ...    }

What I want is for the counter to be updated/incremented atomically. And for a memory release to happen on that address. If I werent using shared memory, for example, it would be something like

std::atomic<int> counter;
atomic_store(counter, newvalue, std::memory_order_release); // perform release     operation on the affected memory location making the write visible to other threads

How do I achieve this for a random memory location (interpreted to be DataBlock counter >above). I can guarantee the address is aligned as required by the architecture (x86 linux)

1. Make the update atomic - how? (i.e. atomicupdate(addr, newvalue))
2. Memory syncing for multicore - (i.e. memorysync(addr)) - only way I can see is using the std::atomic_thread_fence(std::memory_order_release) - but this will "establish memory synchronization ordering of ALL atomic and relaxed atomic stores" - thats overkill for me - I just want the counter location to be synchronized. Appreciate any thoughts.

Answer 1

I can't answer with authority here, but I can give related information that might help.

1. Mutexes can be created in shared memory and/or created to be cross-process. Pthread has a special creation flag, I can't remember if that uses shared memory, or you then share a handle. The linux "futex" can use shared memory directly (note the user address may differ, but the underlying real address should be the same)

2. Hardware atomics work on memory and not process variables. That is, your chip won't care which programs are modifying the variables, the lowest level atomics will thus naturally be cross-process. The same applies to fences.

3. C++11 fails to specify cross-process atomics. However, if they are lock-free (check the flag) it is hard to see how a compiler could implement them such that cross-process wouldn't work. But you'd be placing a lot of faith in your tool-chain and final platform.

4. CPU dependency guarantees also track real memory addresses, so as long as your program would be correct in a threaded form it should also be correct in its multi-process form (with respect to visibility).

5. Kerrek is correct, the abstract machine doesn't really mention multiple processes. However, its synchronization details are written in a way such that they'd equally apply to inter-process as they do to multi-thread. This relates to #3: it'd be hard for a compiler to screw this up.

Short answer, there is no standards compliant way to do this. However, leaning on the way the standard defines mutli-threads there are a lot of assumptions you can make for a quality compiler.

The biggest question is whether an atomic can simply be allocated in shared memory (placement new) and work. Obviously this would only work if it is a true hardware atomic. My guess however is that with a quality compiler/libary the C++ atomics should work find in shared memory.

Have fun verifying behaviour. :)

Answer 2

Since you're on Linux, you can use the gcc atomic built-in __sync_fetch_and_add() on the address for counter ... according to the gcc-documentation on atomic built-ins, this will also implement a full memory fence, not a release operation, but since you actually want a read-modify-write operation rather than simply a load (i.e., incrementing a counter is not just a load, but you have to read, then modify, and finally write-back the value), the full-memory fence is going to be a better choice to enforce the correct memory ordering for this operation.