Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

In Akka.NET, (using Akka.Cluster) how to configure multiple seed nodes (Lighthouse) to know about each other?

If I want to have two Lighthouse seed nodes running on different PCs, where each should know about the other, how should I configure them? Although I have found mention of using multiple Lighthouse instances and seen non-seed node configurations that make use of multiple seed nodes, I have not been able to find an example of multiple seed nodes (specifically Lighthouse) on separate PCs that are aware of one another.

I currently have something like this, but I'm not certain if it is correct.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <configSections>
        <section name="akka" type="Akka.Configuration.Hocon.AkkaConfigurationSection, Akka" />
    </configSections>
    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
    <akka>
        <hocon>
            <![CDATA[
                    lighthouse{
                            actorsystem: "MySystem" #POPULATE NAME OF YOUR ACTOR SYSTEM HERE
                        }

                    akka {
                        actor { 
                            provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
                        }

                        remote {
                            log-remote-lifecycle-events = DEBUG
                            helios.tcp {
                                transport-class = "Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote"
                                applied-adapters = []
                                transport-protocol = tcp
                                #will be populated with a dynamic host-name at runtime if left uncommented
                                #public-hostname = "POPULATE STATIC IP HERE"
                                public-hostname = "192.168.72.55"
                                hostname = "0.0.0.0"
                                port = 4053
                            }
                        }            

                        loggers = ["Akka.Logger.NLog.NLogLogger,Akka.Logger.NLog"]

                        cluster {
                            #will inject this node as a self-seed node at run-time
                            seed-nodes = [
                "akka.tcp://[email protected]:4053"
                "akka.tcp://[email protected]:4053"
              ] #manually populate other seed nodes here, i.e. "akka.tcp://[email protected]:4053", "akka.tcp://[email protected]:4044"
                            roles = [lighthouse]
                        }
                    }
            ]]>
        </hocon>
    </akka>
    <runtime>
        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
            <dependentAssembly>
                <assemblyIdentity name="NLog" publicKeyToken="5120e14c03d0593c" culture="neutral" />
                <bindingRedirect oldVersion="0.0.0.0-4.0.0.0" newVersion="4.0.0.0" />
            </dependentAssembly>
        </assemblyBinding>
    </runtime>
</configuration>

With the configuration of the other instance basically the same, but with these differences:

akka.remote:

public-hostname = "192.168.72.56"

akka.cluster:

    seed-nodes = [
  "akka.tcp://[email protected]:4053"
  "akka.tcp://[email protected]:4053"
] #manually populate other seed nodes here, i.e. "akka.tcp://[email protected]:4053", "akka.tcp://[email protected]:4044"

Is this right? Are there things I should change or could improve? What is the best way to ensure everything is working as it should? Should I remove the first seed-node in each of these configurations, since the comments seem to indicate that including it isn't necessary?

If I start the node on 192.168.72.55 and then the node on 192.168.72.56, I see it join on 192.168.72.55 and receive a welcome message on the 192.168.72.56. If I shutdown the 192.168.72.56 instance and start it again, it is welcomed again, but the 192.168.72.55 instance keeps outputting messages such as:

Akka.Cluster.ClusterCoreDaemon: Leader can currently not perform its duties, rea
chability status: [akka.tcp://[email protected]:4053 -> UniqueAddr
ess: (akka.tcp://[email protected]:4053, 1180787153): Unreachable
[Unreachable] (2), ], member status: [$akka.tcp://[email protected]
:4053 $Up seen=$True, $akka.tcp://[email protected]:4053 $Up seen
=$False]
Akka.Remote.MySystem: Association with remote system akka.tcp:
//[email protected]:4053 has failed; address is now gated for 5000
 ms. Reason is: [Akka.Remote.EndpointDisassociatedException: Disassociated
   at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve
l)
   at Akka.Remote.EndpointWriter.Unhandled(Object message)
   at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti
alAction`1 partialAction)
   at Akka.Actor.ReceiveActor.<>c__DisplayClass11_0.<Become>b__0(Object m)
   at Akka.Actor.ActorCell.<>c__DisplayClass109_0.<Akka.Actor.IUntypedActorConte
xt.Become>b__0(Object m)
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
   at Akka.Actor.ActorCell.ReceiveMessage(Object message)
   at Akka.Actor.ActorCell.ReceivedTerminated(Terminated t)
   at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope)
   at Akka.Actor.ActorCell.Invoke(Envelope envelope)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Akka.Actor.ActorCell.HandleFailed(Failed f)
   at Akka.Actor.ActorCell.SystemInvoke(Envelope envelope)]
Akka.Actor.OneForOneStrategy: Disassociated Akka.Remote.EndpointDisassociatedExc
eption: Disassociated
   at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve
l)
   at Akka.Remote.EndpointWriter.Unhandled(Object message)
   at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti
alAction`1 partialAction)
   at Akka.Actor.ReceiveActor.<>c__DisplayClass11_0.<Become>b__0(Object m)
   at Akka.Actor.ActorCell.<>c__DisplayClass109_0.<Akka.Actor.IUntypedActorConte
xt.Become>b__0(Object m)
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
   at Akka.Actor.ActorCell.ReceiveMessage(Object message)
   at Akka.Actor.ActorCell.ReceivedTerminated(Terminated t)
   at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope)
   at Akka.Actor.ActorCell.Invoke(Envelope envelope)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Akka.Actor.ActorCell.HandleFailed(Failed f)
   at Akka.Actor.ActorCell.SystemInvoke(Envelope envelope)

And later:

Akka.Cluster.ClusterCoreDaemon: Leader can currently not perform its duties, reachability status: [], member status: [$akka.tcp://[email protected]:4053 $Up seen=$True, $akka.tcp://[email protected]:4053 $Up seen =$False]
Akka.Remote.ReliableDeliverySupervisor: Association with remote system akka.tcp:
//[email protected]:4053 has failed; address is now gated for 5000
 ms. Reason is: [Akka.Remote.EndpointDisassociatedException: Disassociated
   at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve
l)
   at Akka.Remote.EndpointWriter.Unhandled(Object message)
   at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti
alAction`1 partialAction)
   at Akka.Actor.ReceiveActor.<>c__DisplayClass11_0.<Become>b__0(Object m)
   at Akka.Actor.ActorCell.<>c__DisplayClass109_0.<Akka.Actor.IUntypedActorConte
xt.Become>b__0(Object m)
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
   at Akka.Actor.ActorCell.ReceiveMessage(Object message)
   at Akka.Actor.ActorCell.ReceivedTerminated(Terminated t)
   at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope)
   at Akka.Actor.ActorCell.Invoke(Envelope envelope)]
Akka.Actor.OneForOneStrategy: Disassociated Akka.Remote.EndpointDisassociatedExc
eption: Disassociated
   at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve
l)
   at Akka.Remote.EndpointWriter.Unhandled(Object message)
   at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti
alAction`1 partialAction)
   at Akka.Actor.ReceiveActor.<>c__DisplayClass11_0.<Become>b__0(Object m)
   at Akka.Actor.ActorCell.<>c__DisplayClass109_0.<Akka.Actor.IUntypedActorConte
xt.Become>b__0(Object m)
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
   at Akka.Actor.ActorCell.ReceiveMessage(Object message)
   at Akka.Actor.ActorCell.ReceivedTerminated(Terminated t)
   at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope)
   at Akka.Actor.ActorCell.Invoke(Envelope envelope)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Akka.Actor.ActorCell.HandleFailed(Failed f)
   at Akka.Actor.ActorCell.SystemInvoke(Envelope envelope)

Doing this again I noticed the following messages:

Akka.Cluster.ClusterCoreDaemon: New incarnation of existing member [UniqueAddres
s: (akka.tcp://[email protected]:4053, 1173825306)] is trying to j
oin. Existing will be removed from the cluster and then new member will be allow
ed to join.
Akka.Cluster.ClusterCoreDaemon: Marking node [akka.tcp://[email protected]
8.72.56:4053] as Down
Akka.Cluster.ClusterCoreDaemon: Leader can perform its duties again

Akka.Cluster.ClusterCoreDaemon: New incarnation of existing member [UniqueAddres
s: (akka.tcp://[email protected]:4053, 253329532)] is trying to jo
in. Existing will be removed from the cluster and then new member will be allowe
d to join.
Akka.Cluster.ClusterCoreDaemon: Cluster Node [akka.tcp://[email protected]
8.72.55:4053] - Marking node(s) as UNREACHABLE [Member(address = akka.tcp://Sup
[email protected]:4053, status = Down]
Akka.Cluster.ClusterCoreDaemon: Leader is removing unreachable node [akka.tcp://
[email protected]:4053]
Akka.Event.DummyClassForStringSources: Association to [akka.tcp://[email protected]:4053] having UID [1180787153] is irrecoverably failed. UID is
now quarantined and all messages to this UID will be delivered to dead letters.
Remote actorsystem must be restarted to recover from this situation.
Akka.Remote.ReliableDeliverySupervisor: Association with remote system akka.tcp:
//[email protected]:4053 has failed; address is now gated for 5000
 ms. Reason is: [Akka.Remote.EndpointDisassociatedException: Disassociated
   at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve
l)
   at Akka.Remote.EndpointWriter.Unhandled(Object message)
   at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti
alAction`1 partialAction)
   at Akka.Actor.ReceiveActor.<>c__DisplayClass11_0.<Become>b__0(Object m)
   at Akka.Actor.ActorCell.<>c__DisplayClass109_0.<Akka.Actor.IUntypedActorConte
xt.Become>b__0(Object m)
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
   at Akka.Actor.ActorCell.ReceiveMessage(Object message)
   at Akka.Actor.ActorCell.ReceivedTerminated(Terminated t)
   at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope)
   at Akka.Actor.ActorCell.Invoke(Envelope envelope)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Akka.Actor.ActorCell.HandleFailed(Failed f)
   at Akka.Actor.ActorCell.SystemInvoke(Envelope envelope)]
Akka.Cluster.ClusterCoreDaemon: Node [akka.tcp://[email protected]
:4053] is JOINING, roles [lighthouse]
Akka.Actor.OneForOneStrategy: Disassociated Akka.Remote.EndpointDisassociatedExc
eption: Disassociated
   at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve
l)
   at Akka.Remote.EndpointWriter.Unhandled(Object message)
   at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti
alAction`1 partialAction)
   at Akka.Actor.ReceiveActor.<>c__DisplayClass11_0.<Become>b__0(Object m)
   at Akka.Actor.ActorCell.<>c__DisplayClass109_0.<Akka.Actor.IUntypedActorConte
xt.Become>b__0(Object m)
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
   at Akka.Actor.ActorCell.ReceiveMessage(Object message)
   at Akka.Actor.ActorCell.ReceivedTerminated(Terminated t)
   at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope)
   at Akka.Actor.ActorCell.Invoke(Envelope envelope)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Akka.Actor.ActorCell.HandleFailed(Failed f)
   at Akka.Actor.ActorCell.SystemInvoke(Envelope envelope)
Akka.Cluster.ClusterCoreDaemon: Leader is moving node [akka.tcp://[email protected]:4053] to [Up]
like image 677
foven Avatar asked Oct 30 '22 07:10

foven


1 Answers

Maybe You should use BroadcastRouter to aggregate each nodes. Than each node should tell somethink like "I am" to this router.

In the effect, each BroadcastRouter's child should receive "I am" with Sender property.

In this way, You should use this constructor: https://github.com/akkadotnet/akka.net/blob/master/src/core/Akka/Routing/Broadcast.cs#L266

like image 140
Paweł Siemienik Avatar answered Nov 17 '22 05:11

Paweł Siemienik