apache-kafkaprotocol-buffersconfluent-cloudkafka-rest

Produce Protobuf message using Rest proxy API


I am exploring options to integrate an ITSM system with Kafka. This needs Protobuf messages sent to Kafka via rest proxy API.

I followed below two articles and set the request body and headers.

https://quachtd.com/using-kafka-rest-proxy-with-protobuf/

https://docs.confluent.io/platform/current/kafka-rest/quickstart.html#produce-and-consume-protobuf-messages

On Confluent, I have created a topic and a schema for testing. Though I am able to produce a message under the topic via JSON, I am not able to produce Protobuf messages.

I am not even able to test the ‘GET’ method as I am not able to find the SchemaRegistryHost and port. How can get these details from the Confluent console? http://{SchemaRegistryHost}:{SchemaRegistryPort}/subjects/salesorder_topic-value/versions/1

Though I took the Schema ID manually and supplied it in the input, I received the message “{“error_code”:405,“message”:“HTTP 405 Method Not Allowed”}”. This made me think if there is any action needed on Schema for the ‘POST’ method to work.

Aside, as this is not JSON, I wonder if I could not test this out in Postman. If it’s possible how can I test it otherwise what other tools can help me to perform the test?

enter image description here


Solution

  • Before adding a schema to the registry you have to set the mode to IMPORT.

    The mode parameter defines which operations are allowed on schemas.

    IMPORT
    READONLY
    READWRITE
    

    The first one allows to import a schema and specify id and version.

    You can set the mode for all subjects or on a specific subject. To enable mode changes on Schema Registry, you have to set the following properties in the Schema Registry properties file before starting it:

    mode.mutability=true

    With curl you might use the command:

    curl -X PUT -H "Content-Type: application/json" http://{SchemaRegistryHost}:{SchemaRegistryPort}/mode --data '{"mode": "IMPORT"}'