[SOLVED] Problem producing Avro serialized object through kafka-avro-console-producer

Problem producing Avro serialized object through kafka-avro-console-producer

I'm producing a message by using the kafka-avro-console-producer binary by doing:

kafka-avro-console-producer --broker-list broker:9092 --topic example-topic --property schema.registry.url='http://schema-registry:8081 --property value.schema='{"type": "record","name": "test","fields": [{"name": "before", "type": ["null", {"type": "record", "name": "columns", "fields":[{"name": "name", "type": "string"}]}],"default": "null"},{"name": "after", "type": ["null", "columns"],"default": "null"}]}'
{"before": null,"after": {"name": "John"}}'

sending the following message:

{"before": null,"after": {"name": "John"}}

and by appling the following Avro schema:

{
    "type": "record",
    "name": "test",
    "fields": [{
        "name": "before",
        "type": ["null", {
            "type": "record",
            "name": "columns",
            "fields": [{
                "name": "name",
                "type": "string"
            }]
        }],
        "default": "null"
    }, {
        "name": "after",
        "type": ["null", "columns"],
        "default": "null"
    }]
}

the error I'm getting in return is the following:

Caused by: org.apache.avro.AvroTypeException: Unknown union branch name
    at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:445)
    at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
    at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
    at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
    at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:178)
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
    at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:240)
    at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
    at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
    at io.confluent.kafka.formatter.AvroMessageReader.jsonToAvro(AvroMessageReader.java:213)
    at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:180)
    at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:55)
    at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)

For those of you that are willing to go deeper into the rabbit hole, I'm making an integration between Oracle Golden Gate and Apache Kafka by using the Oracle Golden Gate Big Data connector. I'm currently experiencing problems with an equivalent model of the one described in here:

https://www.ateam-oracle.com/oracle-goldengate-big-data-adapter-apache-kafka-producer

When trying to apply the schema described in the above webpage to it's corresponding model(and after completing the JSON model), I'm getting the same error as the one I'm getting with the model and schema in the question.

Thank you all very much.

Solution

This is the problem

"type": ["null", "columns"]

You cannot refer back to other record types. You'll need to expand that out like you did for the other field