I have 3 dockers kafka, divolt and streamsets (https://github.com/divolte/docker-divolte) started by compose-up. I want to convert the topic messages to avro files. I created the pipeline in streamset and paste the avro schema, but got an error:
com.streamsets.pipeline.api.base.OnRecordErrorException: KAFKA_37 - Cannot parse record from message 'divolte::3::0': java.io.IOException: Invalid int encoding
at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource$MultiTopicCallable.createRecord(MultiKafkaSource.java:192)
at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource$MultiTopicCallable.sendBatch(MultiKafkaSource.java:158)
at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource$MultiTopicCallable.call(MultiKafkaSource.java:135)
at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource$MultiTopicCallable.call(MultiKafkaSource.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Invalid int encoding
I red that problem is in incorrect avro schema. Could you tell where I can find the correct avro schema for this? I cant find it in dockers and github.
Looks like it might be in the Divolte GitHub repo, at https://github.com/divolte/divolte-schema/blob/master/src/main/resources/DefaultEventRecord.avsc