interlock vs mutex, scale-up issues

Question

I am facing the problem, that I have an C# (.NET) object shared among some threads. A thread might replace the object with another one. The threads are waken up from a TCP/IP connection using the Asynchronous framework.

Sequence:

Threads (waiting for connection) -> Asynchronous Callback -> Do something thread-safe -> Access the shared object -> Do something thread-safe.

1. Solution Mutex:

Object sharedObject = new Object();
Mutex objectMutex = new Mutex();

void threadCallback()
{
  Object newObject = new Object();

  // some processing

  objectMutex.lock();
  // do exchange sharedObject with newObject if needed
  // very little processing here
  objectMutex.unlock();

  // some processing
}

2. Solution Interlock

Object sharedObject = new Object();
int usingSharedObject = 0;

void threadCallback()
{
  Object newObject = new Object();

  // some processing

  // poll until we lock
  while(1 == Interlocked.Exchange(ref usingSharedObject , 1))
  {
    // do exchange sharedObject with newObject if needed
    // very little processing here
    Interlocked.Exchange(ref usingSharedObject , 0); // free lock
  }

  // some processing
}

What is faster and scales up better?

I expect the second solution to be faster as long as there are not to many threads polling at the same time. The second solution might even sleep a random time so that polling does not eat up any processing time. The first solution looks cleaner to me if I really do need to process a lot of TCP/IP connections. Since I do very little processing in the locked section regarding the TCP/IP processing, will there be any scale-up-issues?

How about the object creation at the beginning of the threadCallback() function.

In my C++ background I always used memory-pools in such a situation, since I must use safe .NET is there a fast way to create new Objects or does the .NET platform perform well in this field.

Best regards,

Friedrich

Answer 1

Your interlocked operation is incorrect. A spin-lock usually looks something like this:

int sharedLock = 0;

void callback()
{

do {
 int existingState = Interlocked.CompareExchange(ref sharedLock, 1, 0);
 if (0 == existingState) {
  break;
 }
} while (true);

try
{
 // operate on the shared state
}
finally
{
 int existingState = Interlocked.Exchange(ref sharedLock, 0);
 Debug.Assert (1 == existingState);
}

}

As for the reason's to use one vs. the other, it depends primarily on the kind of operations done while the lock is held. Very short operations (short arithmetic add/substract, simple state flag changes etc) are better suited for a spinlock. Heavy operations (allocations, IO access) cannot occur under spinlock so they have to be done under true mutex.

Answer 2

It appears at first glance that your 2 examples might not be equivalent. It looks to me like your polling solution using Interlocked.Exchange() will drop into the processing loop while something else has claimed the 'homemade semaphore' and will skip over swapping the sharedObject and newObject if the homemade semaphore is claimed. (Unless I'm misunderstanding something, which is quite possible).

Since correctness issues are more important that performance issues, and synchronization primitives can be quite tricky to get correct I'd go with the Mutex solution first and look to another solution if it turned out to be a problem.

Win32 has added a mutex object with a spin count to get the best of both worlds of what you're trying to do here (I think). However, as far as I know it hasn't been exposed in the .NET framework yet.