Why does Akka fail with "IllegalStateException: cannot create children while terminating or terminated" when testing with ScalaTest?

Question

Here is the error I am receiving from Akka:

[debug] Running TaskDef(com.suredbits.core.util.time.TimeUtilUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.address.AddressDAOSystemTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.policy.PolicyHolderDAOUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(rpc.ModesTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.policy.PolicyDAOSystemTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
java.lang.IllegalStateException: cannot create children while terminating or terminated
    at akka.actor.dungeon.Children$class.makeChild(Children.scala:199)
    at akka.actor.dungeon.Children$class.attachChild(Children.scala:41)
    at akka.actor.ActorCell.attachChild(ActorCell.scala:369)
    at akka.actor.ActorSystemImpl.systemActorOf(ActorSystem.scala:551)
    at akka.testkit.TestKitBase$class.$init$(TestKit.scala:125)
    at akka.testkit.TestKit.<init>(TestKit.scala:718)
    at com.suredbits.core.policy.PolicyDAOSystemTest.<init>(PolicyDAOSystemTest.scala:34)
    at com.suredbits.core.policy.PolicyDAOSystemTest.<init>(PolicyDAOSystemTest.scala:46)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at java.lang.Class.newInstance(Class.java:379)
    at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:641)
    at sbt.TestRunner.runTest$1(TestFramework.scala:84)
    at sbt.TestRunner.run(TestFramework.scala:94)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:219)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:219)
    at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:207)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:219)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:219)
    at sbt.TestFunction.apply(TestFramework.scala:224)
    at sbt.Tests$.sbt$Tests$$processRunnable$1(Tests.scala:211)
    at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:217)
    at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:217)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:45)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:45)
    at sbt.std.Transform$$anon$4.work(System.scala:64)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:244)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
[error] Could not run test com.suredbits.core.policy.PolicyDAOSystemTest: java.lang.IllegalStateException: cannot create children while terminating or terminated
[debug] Running TaskDef(com.suredbits.core.policy.PolicyDAOUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.CurrencyUnitsTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.AddressMonitorUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.blockexplorer.BlockCypherSvcUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.address.AddressDAOUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.protocol.BitcoinAddressTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])
[debug] Running TaskDef(com.suredbits.core.blockexplorer.BlockchainInfoSvcUnitTest, org.scalatest.tools.Framework$$anon$1@7e1522e, false, [SuiteSelector])

Here is the test case I am trying to run:

class PolicyDAOSystemTest(actorSystemConfig: ActorSystemConfig) extends TestKit(actorSystemConfig.actorSystem) with ImplicitSender with WordSpecLike
  with MustMatchers with BeforeAndAfter with ScalaFutures with PolicyDAOComponent with PolicyHolderDAOComponent with DbConfigTest with ActorSystemConfig {
  import actorSystemConfig.actorSystem._

  private val dbManagement = new DbManagement with DbConfigTest

  // 1 month
  private val policyDuration = new RichDuration(new Duration(1000 * 60 * 60 * 24 * 30))
  private val policyCreation = new RichDateTime(new DateTime)



  def this() = this(ActorSystemConfig)
}

Here is the scala code for ActorSystemConfig

trait ActorSystemConfig {
  val actorSystem: ActorSystem = ActorSystem("My-Actor-System")
  implicit val timeout = Timeout(2 seconds)
}


object ActorSystemConfig extends ActorSystemConfig

Lastly, here is my source code for PolicyDAO.

    trait PolicyDAOComponent extends CRUDActorComponent { this: PolicyHolderDAOComponent with DbConfig with ActorSystemConfig =>
      import actorSystem._
      def policyDAOActor: ActorRef = actorSystem.actorOf(Props(new PolicyDAO))
      class PolicyDAO extends CRUDActor[Policy, Long] { 
    ... 
      } 
    }

I think I am running into issues sharing ActorSystemConfig across multiple scala test cases. For instance, ActorSystemConfig is shared in PolicyDAOSystemTest and PolicyHolderDAOSystemTest. I am thinking this is what is causing Akka to throw the IllegalStateException. Where exactly is this conflict happening though? I should be receiving a fresh actor everytime I call policyDAOActor since it is going through system.actorOf, correct?

Thanks for the insight.

Answer 1

This exception is usually a consequence of actorSystem.shutdown:

scala> val actorSystem: ActorSystem = ActorSystem("My-Actor-System")
actorSystem: akka.actor.ActorSystem = akka://My-Actor-System

scala> class Aaa extends Actor{ def receive = { case _ => }}
defined class Aaa

scala> actorSystem.actorOf(Props(new Aaa))
res178: akka.actor.ActorRef = Actor[akka://My-Actor-System/user/$a#-842131493]

scala> actorSystem.shutdown

scala> actorSystem.actorOf(Props(new Aaa))
java.lang.IllegalStateException: cannot create children while terminating or terminated
  at akka.actor.dungeon.Children$class.makeChild(Children.scala:200)
  at akka.actor.dungeon.Children$class.attachChild(Children.scala:40)
  at akka.actor.ActorCell.attachChild(ActorCell.scala:369)
  at akka.actor.ActorSystemImpl.actorOf(ActorSystem.scala:554)
  ... 43 elided

As you're sharing one instance of ActorSystem between many tests by default - it's possible that one of them did shutdown explicitly or implicitly (for example inside after or even afterAll). Also, you have created two actor systems with the same name here - one from (actorSystemConfig: ActorSystemConfig), second from - with ActorSystemConfig (see PolicyDAOSystemTest). First one is shared between all tests in application by default.