I am trying to learn akka.net clustering.
I thought I understood that when a node went down, it would be removed from the cluster. But that does not seem to be happening.
I fired up a instance of Lighthouse (as the seed node) and made a super simple Akka.net project and connected then. It all connected fine.
But when I killed the node, Lighthouse keeps looking for it over and over. Eventually it will say something about the Leader not being able to perform its duties.
I know that the node did not leave the cluster gracefully, but I imagine that I will have nodes that crash.
I thought that when that happens, the gossip system was supposed to remove the dead node from the cluster and everything would move on. (Then if the node came back online, it could ask to be added back into the cluster.
But I must be missing something. Because Lighthouse just keeps retrying over and over.
Why does it do that instead of just waiting for it to connect again?
I added this to the "Cluster" part of my configuration and it caused the node to timeout:
auto-down-unreachable-after = 5s