javaspring-bootapache-kafka

Caused by: java.lang.IllegalArgumentException: Failed to load: OrderEvent for OrderEvent Error While consuming Object in Kafka springboot


Caused by: java.lang.IllegalArgumentException: Failed to load: OrderEvent for OrderEvent Error I am getting while consuming Object type topic in my apache kafka consumer.

I am trying to implement sample demo for choreographic Saga pattern using springboot and apache kafka based on event passing. I am trying to pass OrderEvent object from producer to consume. But while I am running the consumer , I am getting the above mentioned error

I am reffering the following links to refer when I got this error. And while reading I got to knew that , this is happening because of mapping need to to explicitly define in my application.properties file

Reffered links

https://docs.spring.io/spring-kafka/reference/kafka/serdes.html#serdes-mapping-types

Kafka consumer ClassNotFoundException

And also adding my application code downside,

application.properties producer side

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=orders-group
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-   serializer=org.springframework.kafka.support.serializer.JsonSerializer
spring.kafka.producer.properties.spring.json.type.mapping=orderEvent:com.order.model.OrderEvent

ProducerService file code like the following

    @Autowired 
    private KafkaTemplate<String, OrderEvent> kafkaTemplate;

    OrderEvent orderEvent =  new OrderEvent();
    orderEvent.oid = orderObj.getOid();
    orderEvent.pid=orderObj.getPid();
    orderEvent.quantity=orderObj.getQuantity();
    orderEvent.mode=orderObj.getMode();
    kafkaTemplate.send("order-create-event", orderEvent);   

Consumer properties file

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=orders-group
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer
spring.kafka.consumer.properties.spring.json.trusted.packages=*
spring.kafka.consumer.properties.spring.json.type.mapping=orderEvent:com.payment.model.OrderEvent 

Consumer Controller Code

@KafkaListener(topics = "order-create-event", groupId = "orders-group")
public void processOrder(OrderEvent orderEventObj) throws JsonMappingException, JsonProcessingException {

    Carts cartObj = new Carts();            
    cartObj.setPid(orderEventObj.pid);
    cartObj.setOid(orderEventObj.oid);
    cartObj.setStatus(orderEventObj.status);
    cartObj.setMode(orderEventObj.mode);
    cartRepo.save(cartObj); 
     // remaining codes
     }

Error

 `Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer`

TrubleShooted Method

 By adding

     `spring.kafka.producer.properties.spring.json.type.mapping` 

and 

 `spring.kafka.consumer.properties.spring.json.trusted.packages=*`

Modified Code And Error

Modified with adding following in producer application.properties

spring.kafka.producer.properties.spring.json.type.mapping=orderEvent:com.order.model.OrderEvent

And added the following with consumer application.properties

spring.kafka.consumer.properties.spring.json.type.mapping=orderEvent:com.payment.model.OrderEvent

enter image description here

I recently started to explore springboot with apache kafka, Can anyone suggest where I went wrong in implementation or suggest any documentation to refer to resolve this issue?


Solution

  • The map is from a class to a token (producer side) and token to class (consumer side).

    Producer :

    spring.kafka.producer.properties.spring.json.type.mapping=OrderEvent:com.order.model.OrderEvent,OrderEvent:OrderEvent
    

    Consumer :

    spring.kafka.consumer.properties.spring.json.type.mapping=OrderEvent:com.payment.model.OrderEvent,OrderEvent:OrderEvent 
    
    

    On the producer side you need:

    spring.kafka.producer.properties.spring.json.type.mapping=orderEvent:com.order.model.OrderEvent
    

    on the consumer side.

    spring.kafka.consumer.properties.spring.json.type.mapping=orderEvent:com.payment.model.OrderEvent