Failing a scalatest when akka actor throws exception outside of the test thread

Question

I've had a situation come up and bite me a few times where I'm testing an Actor and the Actor throws an exception unexpectedly (due to a bug), but the test still passes. Now most of the time the exception in the Actor means that whatever the test is verifying won't come out properly so it the test fails, but in rare cases that's not true. The exception occurs in a different thread than the test runner so the test runner knows nothing about it.

One example is when I'm using a mock to verify some dependency gets called, and due to a mistake in the Actor code I call an unexpected method in the mock. That causes the mock to throw an exception which blows up the actor but not the test. Sometimes this can even cause downstream tests to fail mysteriously because of how the Actor blew up. For example:

// using scala 2.10, akka 2.1.1, scalatest 1.9.1, easymock 3.1
// (FunSpec and TestKit)
class SomeAPI {
  def foo(x: String) = println(x)
  def bar(y: String) = println(y)
}

class SomeActor(someApi: SomeAPI) extends Actor {
  def receive = {
    case x:String  =>
      someApi.foo(x)
      someApi.bar(x)
  }
}

describe("problem example") {
  it("calls foo only when it receives a message") {
    val mockAPI = mock[SomeAPI]
    val ref = TestActorRef(new SomeActor(mockAPI))

    expecting {
      mockAPI.foo("Hi").once()
    }

    whenExecuting(mockAPI) {
      ref.tell("Hi", testActor)
    }
  }

  it("ok actor") {
    val ref = TestActorRef(new Actor {
      def receive = {
        case "Hi"  => sender ! "Hello"
      }
    })
    ref.tell("Hi", testActor)
    expectMsg("Hello")
  }
}

"problemExample" passes, but then downstream "ok actor" fails for some reason I don't really understand... with this exception:

cannot reserve actor name '$$b': already terminated
java.lang.IllegalStateException: cannot reserve actor name '$$b': already terminated
at       akka.actor.dungeon.ChildrenContainer$TerminatedChildrenContainer$.reserve(ChildrenContainer.scala:86)
at akka.actor.dungeon.Children$class.reserveChild(Children.scala:78)
at akka.actor.ActorCell.reserveChild(ActorCell.scala:306)
at akka.testkit.TestActorRef.<init>(TestActorRef.scala:29)

So, I can see ways of catching this sort of thing by examining the logger output in afterEach handlers. Definitely doable, although a little complicated in cases where I actually expect an exception and that's what I'm trying to test. But is there any more direct way of handling this and making the test fail?

Addendum: I have looked at the TestEventListener and suspect there's maybe something there that would help, but I can't see it. The only documentation I could find was about using it to check for expected exceptions, not unexpected ones.

Answer 1

Thinking in Actors there is also another solution: failures travel to the supervisor, so that is the perfect place to catch them and feed them into the test procedure:

val failures = TestProbe()
val props = ... // description for the actor under test
val failureParent = system.actorOf(Props(new Actor {
  val child = context.actorOf(props, "child")
  override val supervisorStrategy = OneForOneStrategy() {
    case f => failures.ref ! f; Stop // or whichever directive is appropriate
  }
  def receive = {
    case msg => child forward msg
  }
}))

You can send to the actor under test by sending to failureParent and all failures—expected or not—go to the failures probe for inspection.

Answer 2

Other than examining the logs, I can think of two ways to fail tests when an actor crashes:

• Ensure that no Terminated message is received
• Check the TestActorRef.isTerminated property

The latter option is deprecated, so I'll ignore it.

Watching Other Actors from Probes describes how to setup a TestProbe. In this case it might look something like:

val probe = TestProbe()
probe watch ref

// Actual test goes here ...

probe.expectNoMessage()

If the actor dies due to an exception it will generate the Terminated message. If that happens during the test and you expect something else, the test will fail. If it happens after your last message expectation, then the expectNoMessage() should fail when Terminated is received.