Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to add headers in kafka-console-producer.sh

I'd like to use the kafka-console-producer.sh to fire a few JSON messages with Kafka headers.

Is this possible?

docker exec -it kafka_1 /opt/kafka_2.12-2.3.0/bin/kafka-console-producer.sh --broker-list localhost:9093 --topic my-topic --producer.config /opt/kafka_2.12-2.3.0/config/my-custom.properties
like image 427
tosi Avatar asked Nov 05 '19 17:11

tosi


People also ask

Where is Kafka console producer sh?

Run Kafka Producer Console Kafka provides the utility kafka-console-producer.sh which is located at ~/kafka-training/kafka/bin/kafka-console-producer.sh to send messages to a topic on the command line.

Is Kafka fire and forget?

Kafka provides fault-tolerance via replication so the failure of a single node or a change in partition leadership does not affect availability. If you configure your producers without acks (otherwise known as "fire and forget"), messages can be silently lost.


1 Answers

No, but you can with kafkacat's -H argument:

Produce:

echo '{"col_foo":1}'|kafkacat -b localhost:9092 -t test -P -H foo=bar

Consume:

kafkacat -b localhost:9092 -t test -C -f '-----\nTopic %t[%p]\nOffset: %o\nHeaders: %h\nKey: %k\nPayload (%S bytes): %s\n'
-----
Topic test[0]
Offset: 0
Headers: foo=bar
Key:
Payload (9 bytes): col_foo:1
% Reached end of topic test [0] at offset 1
like image 92
Robin Moffatt Avatar answered Sep 20 '22 07:09

Robin Moffatt