apache-kafkaconfluent-kafka-dotnet

How to configure a kafka broker in one cluster to publish message


I have two broker and I want to restrict 2nd one with particular messages for particular region.
for example we have multiple region in world to provide our services like china, India
I just wanted to divert specific partner data for India region to broker1 and china region to broker2.
If this possible, I'm looking for solution need to implement in C#, but welcome ideas.

I have initial configuration like this

`"Kafka": {`
    `"Address": "[\"localhost\"]",
`   ` "BrokerHosts": "[\"localhost:9092\"]",`
    `"BrokerHostsIndia": "[\"localhost:9094\"]",`
`}`

Solution

  • Not really sure I understand the problem. Your config here is for clusters, not brokers.

    You can do var bootstrapIndia = config.get("Kafka").get("BrokerHostsIndia"), then set that in ProducerConfig to get a producer for the "India region" cluster.

    One producer cannot send to multiple clusters at once, so you need to have some Dictionary<string, IProducer>, then create multiple instances per-region.

    Or produce to one region, then use tools like MirrorMaker to mirror to another.