Orchestrating cancellation in long-running unit test

Question

I am writing unit tests for the "glue" layer of my application, and am having difficulty creating deterministic tests for asynchronous methods that allow the user to cancel the operation prematurely.

Specifically, in a few async methods we have code that reacts to the cancellation of the call and ensures that the object is in the proper state prior to completing. I would like to ensure that these code paths are covered by tests.

Some C# pseudo code exemplifying a typical async method in this scenario is as follows:

public void FooAsync(CancellationToken token, Action<FooCompletedEventArgs> callback) 
{
   if (token.IsCancellationRequested) DoSomeCleanup0();

   // Call the four helper methods, checking for cancellations in between each
   Exception encounteredException;
   try
   {
      MyDependency.DoExpensiveStuff1();
      if (token.IsCancellationRequested) DoSomeCleanup1();

      MyDependency.DoExpensiveStuff2();
      if (token.IsCancellationRequested) DoSomeCleanup2();

      MyDependency.DoExpensiveStuff3();
      if (token.IsCancellationRequested) DoSomeCleanup3();

      MyDependency.DoExpensiveStuff4();
      if (token.IsCancellationRequested) DoSomeCleanup4();

   }
   catch (Exception e)
   {
      encounteredException = e;
   }

   if (!token.IsCancellationRequested)
   {
      var args = new FooCompletedEventArgs(a bunch of params);
      callback(args);
   }
}

The solution that I have come up with so far involves mocking the underlying MyDependency operations that are wrapped by the glue layer, and forcing each to sleep for an arbitrary period of time. I then invoke the async method, and tell my unit test to sleep for a number of milliseconds before canceling the async request.

Something like this (using Rhino Mocks as an example):

[TestMethod]
public void FooAsyncTest_CancelAfter2()
{
   // arrange
   var myDependency = MockRepository.GenerateStub<IMyDependency>();

   // Set these stubs up to take a little bit of time each so we can orcestrate the cancels
   myDependency.Stub(x => x.DoExpensiveStuff1()).WhenCalled(x => Thread.Sleep(100));
   myDependency.Stub(x => x.DoExpensiveStuff2()).WhenCalled(x => Thread.Sleep(100));
   myDependency.Stub(x => x.DoExpensiveStuff3()).WhenCalled(x => Thread.Sleep(100));
   myDependency.Stub(x => x.DoExpensiveStuff4()).WhenCalled(x => Thread.Sleep(100));

   // act
   var target = new FooClass(myDependency);

   CancellationTokenSource cts = new CancellationTokenSource();
   bool wasCancelled = false;

   target.FooAsync(
      cts.Token,
      args =>
      {
         wasCancelled = args.IsCancelled;
         // Some other code to manipulate FooCompletedEventArgs
      });

   // sleep long enough for two operations to complete, then cancel
   Thread.Sleep(250);
   cts.Cancel();

   // Some code to ensure the async call completes goes here

   //assert
   Assert.IsTrue(wasCancelled);
   // Other assertions to validate state of target go here
}

Aside from the fact that using Thread.Sleep in a unit test makes me queasy, the bigger problem is that sometimes tests like this fail on our build server if it happens to be under significant load. The async call gets too far, and the cancel comes too late.

Can anyone provide a more reliable manner of unit testing cancellation logic for long running operations like this? Any ideas would be appreciated.

Answer 1

I would try to use mocks to "simulate" asynchronous behaviour in a synchronous way. Instead of using

myDependency.Stub(x => x.DoExpensiveStuff1()).WhenCalled(x => Thread.Sleep(100));

and then setting the cancellation flag within whatever number of miliseconds, I would just set it as part of the callback:

myDependency.Stub(x => x.DoExpensiveStuff1());
myDependency.Stub(x => x.DoExpensiveStuff2());
myDependency.Stub(x => x.DoExpensiveStuff3()).WhenCalled(x => cts.Cancel());
myDependency.Stub(x => x.DoExpensiveStuff4());

from your code's perspective this will look as if the cancellation happened during the call.

Answer 2

Each of the long running operations should fire an event when they start running.

Hook this event up in the unit test. This gives deterministic results with the potential that the events could be useful in the future.