akka.netakka.net-cluster

Akka.Net cluster singleton - handover not occurs when current singleton node shutdown unexpectedly


I'm trying Akka.Net Cluster Tools, in order to use the Singleton behavior and it seems to work perfectly, but just when the current singleton node "host" leaves the cluster in a gracefully way. If I suddenly shutdown the host node, the handover does not occur.

Background

I'm building a system that will be composed by four nodes (initially). One of those nodes will be the "workers coordinator" and it will be responsible to monitor some data from database and, when necessary, submit jobs to the other workers. I was thinking to subscribe to cluster events and use the role leader changing event to make an actor (on the leader node) to become a coordinator, but I think that the Cluster Singleton would be a better choice in this case.

Working sample (but just if I gracefully leave the cluster)

private void Start() {
    Console.Title = "Worker";

    var section = (AkkaConfigurationSection)ConfigurationManager.GetSection("akka");
    var config = section.AkkaConfig;

    // Create a new actor system (a container for your actors)
    var system = ActorSystem.Create("SingletonActorSystem", config);
    var cluster = Cluster.Get(system);
    cluster.RegisterOnMemberRemoved(() => MemberRemoved(system));
    var settings = new ClusterSingletonManagerSettings("processorCoordinatorInstance", 
        "worker", TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(1));

    var actor = system.ActorOf(ClusterSingletonManager.Props(
        singletonProps: Props.Create<ProcessorCoordinatorActor>(),
        terminationMessage: PoisonPill.Instance,
        settings: settings),
        name: "processorCoordinator");

    string line = Console.ReadLine();

    if (line == "g") { //handover works
        cluster.Leave(cluster.SelfAddress);
        _leaveClusterEvent.WaitOne();
        system.Shutdown();
    } else { //doesn't work
        system.Shutdown();
    }
}

private async void MemberRemoved(ActorSystem actorSystem) {
    await actorSystem.Terminate();
    _leaveClusterEvent.Set();
}

Configuration

akka {
    suppress-json-serializer-warning = on

    actor {
        provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
    }

    remote {
        helios.tcp {
            port = 0
            hostname = localhost
        }
    }

    cluster {
        seed-nodes = ["akka.tcp://SingletonActorSystem@127.0.0.1:4053"] 
        roles = [worker]
    }
}

Solution

  • Thank you @Horusiath, your answer is totaly right! I wasn't able to find this configuration in akka.net documentation, and I didn't realize that I was supposed to take a look on the akka documentation. Thank you very much!

    Have you tried to set akka.cluster.auto-down-unreachable-after to some timeout (eg. 10 sec)? – Horusiath Aug 12 at 11:27