Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to map types using properties file in Kafka

I have 2 spring boot applications. I am trying to map in the application.properties file in such way that the Consumer can receive the message send by the producer. I want to add that I am using a CustomMessage:

public class CustomMessage {

   private LocalDateTime timestamp;
   private Integer sensor_id;
   private Double measurement_value;
// getters and setters
}

My application.properties file for the producer:

# other properties ( I use CloudKarafka )
spring.kafka.producer.key-serializer= org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer= org.springframework.kafka.support.serializer.JsonSerializer

spring.kafka.producer.properties.spring.json.type.mapping=customMessage:assignment2.kafka.CustomMessage,customMessage:ro.tuc.ds2020.kafkaconsumer.CustomMessage

And for consumer:

spring.kafka.consumer.key-deserializer= org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer= org.springframework.kafka.support.serializer.JsonDeserializer
spring.kafka.consumer.properties.spring.json.type.mapping=customMessage:ro.tuc.ds2020.kafkaconsumer.CustomMessage, customMessage:assignment.kafka.CustomMessage

I am not sure how the type.mapping should be done. Doing in this way presented, I get the following error:

Failed to construct kafka consumer
Failed to load: assignment.kafka.CustomMessage for  customMessage

It is also failing to construct producer.

like image 531
Gloria Avatar asked Nov 15 '25 10:11

Gloria


1 Answers

The map is from a class to a token (producer side) and token to class (consumer side).

You are mapping 2 different types to the same token.

Producer:

customMessage:assignment2.kafka.CustomMessage,
customMessage:ro.tuc.ds2020.kafkaconsumer.CustomMessage

Consumer:

customMessage:ro.tuc.ds2020.kafkaconsumer.CustomMessage, 
customMessage:assignment.kafka.CustomMessage

Assuming the producer's class is assignment2.kafka.CustomMessage you just need

customMessage:assignment2.kafka.CustomMessage

on the producer side and

customMessage:ro.tuc.ds2020.kafkaconsumer.CustomMessage

on the consumer side.

like image 144
Gary Russell Avatar answered Nov 18 '25 03:11

Gary Russell



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!