Multithreaded code makes Rhino Mocks cause a Deadlock

Question

We're currently facing some issues during Unit Testing. Our class is multithreading some function calls on Mocked objects using Rhino Mocks. Here's a example reduced to the minimum:

public class Bar
{
    private readonly List<IFoo> _fooList;

    public Bar(List<IFoo> fooList)
    {
        _fooList = fooList;
    }

    public void Start()
    {
        var allTasks = new List<Task>();
        foreach (var foo in _fooList)
            allTasks.Add(Task.Factory.StartNew(() => foo.DoSomething()));

        Task.WaitAll(allTasks.ToArray());
    }
}

The Interface IFoo is defined as:

public interface IFoo
{
    void DoSomething();
    event EventHandler myEvent;
}

To reproduce the deadlock, our unittest does the following: 1. create some IFoo Mocks 2. Raise myEvent when DoSomething() gets called.

[TestMethod]
    public void Foo_RaiseBar()
    {
        var fooList = GenerateFooList(50);

        var target = new Bar(fooList);
        target.Start();
    }

    private List<IFoo> GenerateFooList(int max)
    {
        var mocks = new MockRepository();
        var fooList = new List<IFoo>();

        for (int i = 0; i < max; i++)
            fooList.Add(GenerateFoo(mocks));

        mocks.ReplayAll();
        return fooList;
    }

    private IFoo GenerateFoo(MockRepository mocks)
    {
        var foo = mocks.StrictMock<IFoo>();

        foo.myEvent += null;
        var eventRaiser = LastCall.On(foo).IgnoreArguments().GetEventRaiser();

        foo.DoSomething();
        LastCall.On(foo).WhenCalled(i => eventRaiser.Raise(foo, EventArgs.Empty));

        return foo;
    }

The more Foo's are generated, the more often the deadlock occurs. If the test won't block, run it several times, and it will. Stopping the debugging testrun shows, that all Tasks are still in TaskStatus.Running and the current worker thread is breaking at

[In a sleep, wait, or join]
Rhino.Mocks.DLL!Rhino.Mocks.Impl.RhinoInterceptor.Intercept(Castle.Core.Interceptor.IInvocation invocation) + 0x3d bytes

The weird thing which confuses us most is the fact, that the signature of the Intercept(...) Method is defined as Synchronized - but several Threads are located here. I've read several postings about Rhino Mocks and Multithreaded, but havn't found warnings (expected setting up the records) or limitations.

 [MethodImpl(MethodImplOptions.Synchronized)]
    public void Intercept(IInvocation invocation)

Are we doing something completely wrong on setting up our Mockobjects or using them in a multithreaded environment? Any help or hint is welcome!

Answer 1

This is a race condition in your code and not a bug in RhinoMocks. The problem occurs when you are setting up the allTasks task list in the Start() method:

public void Start() 
{ 
    var allTasks = new List<Task>(); 
    foreach (var foo in _fooList) 
        // the next line has a bug
        allTasks.Add(Task.Factory.StartNew(() => foo.DoSomething())); 

    Task.WaitAll(allTasks.ToArray()); 
} 

You need to pass the foo instance explicitly into the task. The task will execute on a different thread and it's very likely that the foreach loop will replace the value of foo before the task has started.

This means that each foo.DoSomething() is being invoked sometimes never and sometimes more than once. For this reason, some of the tasks will block indefinitely because RhinoMocks can't handle overlapped raising of events on the same instance from different threads and it gets into a deadlock.

Replace this line in your Start method:

allTasks.Add(Task.Factory.StartNew(() => foo.DoSomething())); 

With this:

allTasks.Add(Task.Factory.StartNew(f => ((IFoo)f).DoSomething(), foo));

This is a classic bug that is subtle and very easy to overlook. It is sometimes referred to as "accessing a modified closure".

PS:

Following the comments on this post, I rewrote this test using Moq. In this case it doesn't block - but beware that expectations created on a given instance might not be satisfied unless the original bug is fixed as described. GenerateFoo() using Moq looks like this:

private List<IFoo> GenerateFooList(int max)
{
    var fooList = new List<IFoo>();

    for (int i = 0; i < max; i++)
        fooList.Add(GenerateFoo());

    return fooList;
}

private IFoo GenerateFoo()
{
    var foo = new Mock<IFoo>();
    foo.Setup(f => f.DoSomething()).Raises(f => f.myEvent += null, EventArgs.Empty);
    return foo.Object;
}

It's more elegant than RhinoMocks - and clearly more tolerant of multiple threads raising events on the same instance simultaneously. Although I don't imagine this is a common requirement - personally I don't often find scenarios where you can assume the subscribers to an event are thread-safe.