We have 3 ignite server nodes in 3 different server farms, full replicated, persistence enabled, and all servers area baseline nodes. It happens that if 2 server nodes fail (node or connection crash or slow connection), the remainig one also perform a shutdown, perhaps guessing it's disconnected from the network. Is it possible to make the surviving node not to shutdown? Is it possible to adjust some timeout to avoid disconnections from slow networks or nodes?
I cannot find any hint into the documentation. To avoid the problem I've to run only one server node (what we tried to avoid using Ignite...).
You can try to customize StopNodeOrHaltFailureHandler https://stackoverflow.com/questions/tagged/ignite with SEGMENTATION added to ignoredFailureTypes. But in this case, if all 3 nodes are segmented and remain alive, you need to keep in mind that the cluster may enter the split-brain state. To decide wich node should be used for cache operations, you can add TopologyValidator https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/TopologyValidator.html to cache config. And based on node attributes to decide which node is allowed.