apache-kafkaapache-kafka-connectconfluent-cloud

Drop list of fields from confluent <-> elastic connector

Using confluent cloud's elastic search connector to push data into elastic. Is there a way to drop/pick only a list of fields ?

I read about transform option available in the connector, but couldnt find any to filter out or drop a list of fields from incoming event.

Solution

  • The ReplaceField transform allows you to exclude/include particular fields.

    If that's not available, the alternative is to use Kafka Streams/KsqlDB to create a new topic that the sink connector will read