Menu
I am trying to send message through kafka console producer. But I am not able to type messages more than 4095 characters. Tried to search if there is any property related to this in producer or server but to not avail. Even tried to search if there is any OS specific limitation or stdin character limitation but did not find anything.
Please help in sending large message through console producer.
658
The Kafka max message size is 1MB. In this lesson we will look at two approaches for handling larger messages in Kafka. Kafka has a default limit of 1MB per message in the topic.
Kafka Broker Configuration An optional configuration property, “message. max. bytes“, can be used to allow all topics on a Broker to accept messages of greater than 1MB in size. And this holds the value of the largest record batch size allowed by Kafka after compression (if compression is enabled).
Out of the box, the Kafka brokers can handle messages up to 1MB (in practice, a little bit less than 1MB) with the default configuration settings, though Kafka is optimized for small messages of about 1K in size.
I found an alternate to do this. Add a file with your input and then send it to the producer. Use the following command:
cat yourFile.xml | kafka-console-producer --broker-list localhost:9092 --topic TopicName
It helps if the contents of your file are in single line. Hopefully, this helps.
145
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
