Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to send key, value messages with the kafka console producer

Tags:

I got a use case where I need to send key value messages with Kafka Console Producer. So how to achieve this through the Kafka Console Producer command?

like image 374
Keshav Lodhi Avatar asked May 28 '20 16:05

Keshav Lodhi


People also ask

How do I use the message key in Kafka?

To write the data and pass message key values to Kafka, add a Kafka Producer destination, and then: On the Data Format tab of the destination, select String for the Message Key Format property.


1 Answers

I found out the solution after some research and the solution is here.

kafka-console-producer command

kafka-console-producer.sh --broker-list localhost:9092 --topic topic-name --property "parse.key=true" --property "key.separator=:"

After running this command you will enter in producer console and from there you can send key, value messages.

For example

key1:value1
key2:value2
key3:value3

For more clarity, I am providing sample key-value message here, emp_info is a key and JSON object is a value.

emp_info: {"emp_id":100,"first_name":"Keshav","last_name":"Lodhi","designation":"DataEngineer"}

Note: Simply sending lines of text will result in messages with null keys. In order to send messages with both keys and values you must set the parse.key and key.separator properties on the command line when running the producer.

like image 144
Keshav Lodhi Avatar answered Sep 17 '22 18:09

Keshav Lodhi