apache-kafkakafka-consumer-apireliability

A topic has 3 replicas and min.insyinc.replicas to 2, what happens when both the follower and in-syinc replica go down and the consumer wants to read?


I am preparing for the CCDAK certification and I stepped into this question:

"A topic has three replicas and you set min.insync.replicas to 2. If two out of three replicas are not available, what happens when a consume request is sent to broker?"

The given answer is:

Data will be returned by the remaining in-sync replica

I had a doubt over a corner case: what happens when both the leader and the in-sync replica go down at the same time (third replica still not in-sync) and the consumer wants to read a message not present in the third replica (it may has not been copied yet since it is a not in-sync replica)?

Would a practical example be something like this: let's say consumer has committed offset 10 and wants to read from 11 but third replica has messages up to offset 9. Leader and follower are down, what would happen? Will the consumer just wait for the other brokers or will Kafka throw an error I have to deal with?


Solution

  • Your consumer will not be rebalanced because from Kafka side, It can not select a leader for that partition. The partition is not Synced and then Kafka will wait to start of other replicas to select a leader. In this scenario, Your consumer will hang.