IIDR CDC Kafka subscription
For an enterprise wide data streaming platform implementation, we are using a confluent kafka and ibm iidr cdc .
Why the IIDR Management console has only two option to use zookeeper & rest proxy while configuring the subscription for kafka ?
From security perspective , it is not recommended at all to expose zookeeper . How can we just provide them broker endpoints to connect to kafka ? Or how can we implement IIDR confluent kafka connector ? https://www.confluent.io/connector/ibm-data-replication/
How SSL can be implemented in IIDR CDC and Kafka ? any document links ?
Thanks!
Sorry for the delay...
The following document describes several scenarios of IDR Kafka configuration. You can provide broker endpoints editing the /conf/kafkaproducer.properties (on the machine you installed IDR Kafka Agent). If you do so, you still have to provide hostname and port number on MC GUI, but it will ignore them (you can provide fake data in those fields). It also explain some considerations on kerberized environments and on SSH connections.
Keep in mind that JRE provided with CDC has restricted security policy files, so, for using keys greater than 56 bytes (i.e. always), you need to download and upgrade them with unrestricted ones (they can be downloaded from IBM FixCentral page).
I hope this helps.
- Spring Cloud Stream Multi Cluster and Multi Input Bindings from a Single Consumer
- How docker map env variable to kafka broker variable?
- Can't migrate old data from zookeeper cluster to KRaft cluster via MirrorMaker2
- Handling bad messages using Kafka's Streams API
- How can I access Confluent Cloud Schema Registry using KafkIO?
- Running Kafka on Windows 10 fails: The system cannot find the path specified
- Is key required as part of sending messages to Kafka?
- Why do we need Outbox pattern with Kafka
- Get the latest offsets in SSL Enabled Kafka via CMD
- Kafka: Consumer API vs Streams API
- Understanding Kafka Topics and Partitions
- Spring Kafka, overriding max.poll.interval.ms?
- KafkaAvroSerializer for serializing Avro without schema.registry.url
- Transactional outbox distributed lock fencing confusion
- Ad-Hoc Snapshot with Debezium SQL Server Source Connector using Kafka Signal doesn't work
- Need to run kafka-storage.sh on every broker and controller
- Consumer gives "This server does not host this topic-partition" error
- Kafka producer callback Exception
- ModuleNotFoundError: No module named 'kafka.vendor.six.moves' in Dockerized Django Application
- problems with Schema Registry associated with Azure Event Hub from KafkaConnect using Strimzi
- Kafka Streams state store vs MongoDB for state management
- Handling entity merges in Kafka when the events for both entities are across two separate partitions?
- Setting Schema Registry Proxy URL and Port
- Apache Kafka metadata fetch takes more than 40 sec to read and fetch why?
- Kafka Consumers leave some messages unread
- kafka mysql source connect errorKafka Connect JDBC Source Connector Fails to Start: IllegalAccessError due to ClassLoader Issue
- org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms
- Error when creating sink connector in MSK Connect: java.lang.NoSuchMethodError: 'boolean org.apache.kafka.common.utils.Utils.isBlank(java.lang.String)
- Classes for Avro serialization not recognized in converter plugins, using Strimzi to deploy KafkaConnect container
- How do I run debezium with Confluent platform?