Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Akka cluster detecting Quarantined state

How can I detect Quarantined state from within the system being quarantined~?

I'm seeing this log below:

[warn] Remoting - Tried to associate with unreachable remote address [akka.tcp://[email protected]:6000]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: The remote system has quarantined this system. No further associations to the remote system are possible until this system is restarted.

but I'm not sure how I can react to this from code.


I found this thread: Recovering from the Quarantined state, suggesting to listen for the QuarantinedEvent but that is not dispatched on the system being quarantined.

And I actually listened for all RemotingLifecycleEvents and found this:

AssociationError [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:6000]: Error [Invalid address: akka.tcp://[email protected]:6000] [akka.remote.InvalidAssociation: Invalid address: akka.tcp://[email protected]:6000 Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system has quarantined this system. No further associations to the remote system are possible until this system is restarted.]

but that is just an AssociationError which will be dispatched for many other reasons as well, do I have to search for the actual text "The remote system has quarantined this system." within the error to be sure??

like image 241
Chris Avatar asked Sep 09 '15 04:09

Chris


1 Answers

Yes, what you suggested works and can be done as follows

Subscribe an actor to akka.remote.AssociationErrorEvent

override def preStart(): Unit = {
  context.system.eventStream.subscribe(self, classOf[akka.remote.AssociationErrorEvent])
}

and then do following in the receive method

override def receive: Receive = {
  case e:AssociationErrorEvent =>
    log.info(s"AssociationErrorEvent: $e")
    if (e.cause.getCause.getMessage.contains("quarantined this system")) {
      log.warning(s"We got quarantined")
    }
}
like image 190
Ivan Stanislavciuc Avatar answered Oct 25 '22 15:10

Ivan Stanislavciuc