Whose responsibility is creating Akka actor's children when it fails?

Question

Akka's documentation correct quote says:

the precise sequence of events during a restart is the following: Suspend the actor (which means that it will not process normal messages until resumed), and recursively suspend all children.

A misleading quote says:

Resuming an actor resumes all its subordinates, restarting an actor entails restarting all its subordinates, similarly terminating an actor will also terminate all its subordinates

What I suspect is that it is our responsibility to create child(ren) in the preStart method because Akka's term RESTART is no reSTART unless you create the children recursively and explicitly in their parent preStart() method.

Example (using Akka 2.0 & 2.2-SNAPSHOT): no matter what I try, children are always just stopped, never restarted in this test case. I create Supervisor -> First -> Second relationship and throw exception in the Supervisor. What happens is that supervisor is restarted and First & Second stopped.

  test("restart test") {
    val system = ActorSystem("test")
    val supervisor = system.actorOf(Props(new Supervisor), "supervisor")
    supervisor ! CREATE(Props(new First), "first")
    Thread.sleep(500)

    val first = system.actorFor("akka://test/user/supervisor/first")
    first ! CREATE(Props(new Second), "second")
    Thread.sleep(500)
    supervisor ! WTF

    Thread.sleep(20000)
  }

  case object WTF
  case class CREATE(p: Props, name: String)

  class Supervisor extends Actor {
    override val supervisorStrategy =
      OneForOneStrategy(maxNrOfRetries = 10) {
        case _: IllegalStateException       => Restart
        case _: IllegalArgumentException    => Stop
        case _: Exception                   => Restart
    }
    override def preStart() {
      println(s"$self starts")
    }
    override def postStop() {
      println(s"$self stopped")
    }
    override def receive = {
      case WTF => println("throwing exception"); throw new IllegalStateException()
      case CREATE(p, name) => context.actorOf(p, name)
    }
  }
  class First extends Actor {
    override def preStart() {
      println(s"$self starts")
    }
    override def postStop() {
      println(s"$self stopped")
    }
    override def receive = {
      case WTF => println("throwing exception"); throw new IllegalStateException()
      case CREATE(p, name) => context.actorOf(p, name)
    }
  }
  class Second extends Actor {
    override def preStart() {
      println(s"$self starts")
    }
    override def postStop() {
      println(s"$self stopped")
    }
    override def receive = {
      case WTF => println("throwing exception"); throw new IllegalStateException()
      case CREATE => sender ! "ok"
    }
  }

Actor[akka://test/user/supervisor#1599926629] starts Actor[akka://test/user/supervisor/first#2012011668] starts Actor[akka://test/user/supervisor/first/second#1750038710] starts

throwing exception Actor[akka://test/user/supervisor#1599926629] stopped [ERROR] [06/26/2013 11:11:16.899] [test-akka.actor.default-dispatcher-4] [akka://test/user/supervisor] null java.lang.IllegalStateException at com.fg.mail.smtp.IntegrationSuite$Supervisor$$anonfun$receive$1.applyOrElse(IntegrationSuite.scala:40) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) at akka.actor.ActorCell.invoke(ActorCell.scala:456) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) at akka.dispatch.Mailbox.run(Mailbox.scala:219) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

Actor[akka://test/user/supervisor/first/second#1750038710] stopped Actor[akka://test/user/supervisor/first#2012011668] stopped Actor[akka://test/user/supervisor#1599926629] starts

Answer 1

Your suspicion is correct. If you look carefully at the 7-step description of the restart process on http://doc.akka.io/docs/akka/snapshot/general/supervision.html, you will see:

2 . ... defaults to sending termination requests to all children ...

and

6 . send restart request to all children which were not killed

So, you need to either override the parent's preRestart hook to stop Akka killing the children, or override postRestart to re-create all the children that were just killed.

Which you choose really depends on the semantics of your application. Sometimes it's useful to kill the entire hierarchy and start with a blank slate, sometimes not.