Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring Kafka JsonSerializer usage

I am trying to follow the instructions here:

http://docs.spring.io/spring-kafka/docs/1.1.1.RELEASE/reference/htmlsingle/#_serialization_deserialization_and_message_conversion

To set up a KafkaTemplate that can serialize and send some simple Java POJOs that I have. But I found the documentation vague and confusing, especially this part:

For this purpose Spring for Apache Kafka also provides JsonSerializer/JsonDeserializer implementations based on the Jackson JSON processor. When JsonSerializer is pretty simple and just lets to write any Java object as a JSON byte[]

...

Although Serializer/Deserializer API is pretty simple and flexible from the low-level Kafka Consumer and Producer perspective, it is not enough on the Messaging level, where KafkaTemplate and @KafkaListener are present.

...

The MessageConverter can be injected into KafkaTemplate instance directly and via AbstractKafkaListenerContainerFactory bean definition for the @KafkaListener.containerFactory() property

So my question is:

  • What is the type of my KafkaTemplate? Is it KafkaTemplate<String, Object>? Or is it KafkaTemplate<String, String>?
  • What is my Serializer class? Is it StringSerializer, or is it JsonSerializer?
  • Do I use kafkaTemplate.setMessageConverter(new StringJsonMessageConverter()) when creating my KafkaTemplate bean?

Apologies if these are stupid questions - I'm trying to understand the correct way of setting it up rather than "hacking it till it kinda works".

like image 857
Matt Avatar asked Nov 01 '16 12:11

Matt


People also ask

Is KafkaTemplate send asynchronous?

The KafkaTemplate wraps a producer and provides convenience methods to send data to kafka topics. Both asynchronous and synchronous methods are provided, with the async methods returning a Future .

What is key deserializer in Kafka?

Every Kafka Streams application must provide SerDes (Serializer/Deserializer) for the data types of record keys and record values (e.g. java. lang. String ) to materialize the data when necessary. Operations that require such SerDes information include: stream() , table() , to() , through() , groupByKey() , groupBy() .

What is key serializer and value serializer in Kafka?

The key. serializer and value. serializer instruct how to turn the key and value objects the user provides with their ProducerRecord into bytes. You can use the included ByteArraySerializer or StringSerializer for simple string or byte types.


1 Answers

  1. <String, Object>

  2. JsonSerializer

  3. The message converter is only used when using the send that takes a Message<?> and with a JsonSerializer you should use the default one.

like image 137
Gary Russell Avatar answered Sep 25 '22 08:09

Gary Russell