apache-zookeeperdistributed-computingconsistent-hashing

Zookeeper for Consistent hashing?


Can we use zookeeper for implementing consistent hashing? I want to maintain my hashring of servers with zookeeper so that their health can be monitored and the logic of mapping a given key to particular server with zookeeper. But since zookeeper needs a leader(as I read), then who can be a leader here in my case?Is it possible to have leader-less set up of zookeeper just for health monitoring?Do I need a leader?

Here is flow I am thinking of :

  1. Registers all servers of hashring with zookeeper.
  2. Client sends key to get the server and zookeeper has list of hosts and it can decide which shard will handle it.(The logic may be hashing based sharding or range base resides with zookeeper)
  3. In case servers added/removed the responsibilities will be adjusted.(I assume we have to manually update responsibilities when a new server added in case if we use range base sharding)?

Solution

  • No, you do not need a leader what you think. Zookeeper will evict a leader to process requests by itself, based on the consensus algorithm ZAB.

    Moreover, you could use ZK to manage a list of nodes, known as group membership