I am using the following connector to get data from MongoDB to my Kafka topic:
{
"class": "com.mongodb.kafka.connect.MongoSourceConnector",
"type": "source",
"version": "1.10.0"
}
And the record coming to the Kafka topic has empty header information:
headers: {}
What I need here is to add a hard-coded header information in the connector, so that all records processed by the connector can have header similar to:
headers: {"Source": "MongoDB"}
I come across a transformation called InsertHeader: https://docs.confluent.io/platform/current/connect/transforms/insertheader.html#insertheader that seems to do exactly what I need, however, when I try to use it with my connector, it is throwing the error:
Invalid value org.apache.kafka.connect.transforms.InsertHeader for configuration transforms.InsertHeader.type: Class org.apache.kafka.connect.transforms.InsertHeader could not be found.
It is stated that InsertHeader is not currently available for managed connectors, is this the reason why it is not useable?
And what are the alternatives that I can use?
Thank you in advance.
When you say "manged connector", I assume you are referring to Confluent Cloud.
In that case, you need to self manage to get control over what plugins/transforms are available for any connector to use
Alternatively, deploy without transforms. Use other tools to consume and insert headers, and write to new topics