Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

kafka-console-producer and bash script

Want to send some message by bash script.

  bin/zookeeper-server-start.sh config/zookeeper.properties > zookeeper.log 2>&1 & sleep 2  bin/kafka-server-start.sh config/server.properties > server.log 2>&1 & sleep 2  #Create topic bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic $1 sleep 2  #Get topics list echo "kafka has next topics:" bin/kafka-topics.sh --list --zookeeper localhost:2181  #send message echo "will send messages:" bin/kafka-console-producer.sh --broker-list localhost:9092 --topic $1 "test 1" 

Kafka started well. And I can send message by the producer console

 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic $1 "test 1" 

But I can't send message into bash script. How can I send it by bash script ?

Thanks.

like image 719
scala Avatar asked Aug 01 '16 14:08

scala


People also ask

Where is Kafka console producer sh?

Run Kafka Producer Console Kafka provides the utility kafka-console-producer.sh which is located at ~/kafka-training/kafka/bin/kafka-console-producer.sh to send messages to a topic on the command line.

How do I open Kafka console?

Step 1: Start the zookeeper as well as the kafka server initially. Step2: Type the command: 'kafka-console-consumer' on the command line. This will help the user to read the data from the Kafka topic and output it to the standard outputs.

Where are Kafka shell scripts?

When you extract the Apache Kafka distribution, the bin/ directory (or the bin\windows\ directory if you're using Windows) of the distribution contains a set of shell scripts that enable you to interact with your Kafka instance.


2 Answers

Try doing this:

echo "test 1" | bin/kafka-console-producer.sh --broker-list localhost:9092 --topic $1 

or this:

cat file.txt | bin/kafka-console-producer.sh --broker-list localhost:9092 --topic $1 
like image 121
serejja Avatar answered Sep 20 '22 18:09

serejja


You can also do the the below. Ignore the sleep parameter.

for x in {1..100}; do echo $x; sleep 2; done | path/to/bin/kafka-console-producer --broker-list <brk:port> --topic <topic_name> 
like image 39
AtulyaB Avatar answered Sep 18 '22 18:09

AtulyaB