IPC through shared memory with atomic_t; is it good for x86?

Question

I have the following code for interprocess communication through shared memory. One process writes to a log and the other reads from it. One way is to use semaphores, but here I'm using atomic flag (log_flag) of type atomic_t which resides inside the shared memory. The log (log_data) is also shared.

Now the question is, would this work for x86 architecture or do I need semaphores or mutexes? What if I make log_flag non-atomic? Given x86 has a strict memory model and proactive cache coherence, and optimizations are not applied on pointers, I think it would still work?

EDIT: Note that I have a multicore processor with 8 cores, so I don't have any problem with busy waits here!

// Process 1 calls this function
void write_log( void * data, size_t size )
{
    while( *log_flag )
           ;
    memcpy( log_data, data, size );
    *log_flag = 1;
}

// Process 2 calls this function
void read_log( void * data, size_t size )
{
    while( !( *log_flag ) )
       ;
    memcpy( data, log_data, size );
    *log_flag = 0;
}

Answer 1

You may want to use the following macro in the loop, to avoid stressing the memory bus:

#if defined(__x86_64) || defined(__i386)
#define cpu_relax() __asm__("pause":::"memory")
#else
#define cpu_relax() __asm__("":::"memory")
#endif

Also, it acts as a memory barrier ("memory" param.), so no need to declare log_flag as volatile.

But I think this is overkill, it should only be done for hard real-time stuff. You should be fine using a futex. And maybe you could simply use a pipe, it's sufficiently fast for almost all purposes.

Answer 2

I wouldn't recommend that for two reasons: first, although pointer access may not be optimized by the compiler, that doesn't mean the pointed value won't be cached by the processor. Second, the fact that it is atomic won't prevent a read access between the end of the while loop and the line that does *log_flag=0. A mutex is safer, though a lot slower.

If you're using pthreads, consider using an RW mutex to protect the whole buffer, that way you don't need a flag to control it, the mutex is itself the flag and you'll have better performance when doing frequent reads.

I also don't recommend doing empty while() loops, you'll hog all the processor that way. Put a usleep(1000) inside the loop to give the processor a chance to breathe.

Answer 3

There are a whole bunch of reasons why you should use a semaphore and not rely on a flag.

1. Your read log while loop is spinning unnnecessarily. This consumes system resources like power unnecessarly. It also means that the CPU cannot be used for other tasks.
2. I will be surprised if x86 fully guarantees read and write ordering. incoming data may set log flag to 1 only to have outgoing data set it to 0. This may potentially mean that you end up losing data.
3. I don't know where you got it from that optimizations are not applied on pointers as a general use. Optimization can be applied anywhere where there is no difference to external change. The compiler will probably not know that log_flag can be changed by a concurrent process.

Problem 2 might appear very may appear rarely and tracking down the issue will be hard. So do yourself a favour and use the correct operating system primitives. They will guarantee that things work as expected.