Hazelcast: how to ensure cluster startup is finished

Question

I have a cluster with 3 nodes (in different machines) and I have a "business logic" that use a distributed lock at startup.

Sometimes when there is more latency every node acquires the exclusive lock with success because the cluster isn't already "startup" so each node does not yet see the other.

Subsequently the nodes see each other and the cluster is correctly configured with 3 nodes. I know there is a "MemberShipListener" to capture the event "Member added" so I could execute again the "business logic", but I would to know if there is a method to ensure when the cluster startup is properly finished in order to wait to execute the "business logic" until the cluster is on.

I tried to use hazelcast.initial.wait.seconds but configure the right seconds isn't deterministic and I don't know if this also delay the member join operations.

Answer 1

Afaik, there is no such thing in Hazelcast. As the cluster is dynamic, a node can go and leave at any time, so the cluster is never "complete" or not.

You can, however :

1. Configure an initial wait, like you described, in order to help with initial latencies
2. use hazelcast.initial.min.cluster.size to define the minimum number of members hazelcast is waiting for at start
3. Define a minimal quorum : the minimal number of nodes for the cluster to be considered as useable/healthy (see cluster quorum)
4. Use the PartitionService to check is the cluster is safe, or if there are pending migrations