apache-kafkaconfluent-schema-registryprotobuf-java

Register dynamic protobuf schemas in kafka schema registry generally


I implemented many pipelines that produce proto typed data to kafka. I want to see if there is a general way to register every proto file that uses different schema with the same function without converting proto message to corresponding java object such as using complier:

  protoc -I=. --java_out=. my_proto.proto

In another words, is there any way to register schema only via .proto file just like registering the avro format via avsc file but that with json format?


Solution

  • There is not. The Schema Registry has a well-defined JSON-based API, and does not accept other schema formats.

    https://docs.confluent.io/platform/current/schema-registry/develop/api.html

    You could try using the Registry maven plugin, though. Otherwise, you need to write your own script to JSONify the proto schema into a payload the registry accepts (keep in mind, this happens automatically when you use ProtobufSerializer from Confluent)