apache-kafkareplicationapache-kafka-mirrormaker

Topic explosion in mirrormaker 2


Does mirrormaker 2 need 6 system topics (on both sides of the wire) for each topic to be replicated? I am concerned this could lead to topic explosion. we need two-way sync between kafka clusters


Solution

  • Three of those topics are for Kafka Connect, not MirrorMaker2 (configs, offsets, and status}.

    No, none of those are unique to each mirrored topic, but rather the Kafka Connect cluster and its group id. You can run multiple mirrors in the same Connect cluster, thereby not creating new set of internal topics.