My project is using Kafaka 0.10.1.0, and I was pawing through the source trying to learn a bit.
I had a question about Kafka's protocol for the exchange of group related metadata, is this a peer-to-peer type protocol wherein the Coordinator for a given consumer group runs on client machines?
The documentation for org.apache.kafka.clients.consumer.internals.AbstractCoordinator mentions one step in the protocol is:
<li>Group/Leader Selection: The coordinator select the members of the group and chooses one member
as the leader.</li>
So, it seems the group leader logic runs on the client side. I guess this would be good for scalability, offloading from the broker, but I wanted to confirm with a Kafa expert that I came to the correct conclusion. Thanks !
One of the brokers is a coordinator for the cluster, and the first consumer client to join a group becomes group coordinator.
By definition of having a single coordinator, it is leader-follower protocol, not p2p, in which each node is typically equal