Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Consumer not receiving messages, kafka console, new consumer api, Kafka 0.9

I am doing the Kafka Quickstart for Kafka 0.9.0.0.

I have zookeeper listening at localhost:2181 because I ran

bin/zookeeper-server-start.sh config/zookeeper.properties 

I have a single broker listening at localhost:9092 because I ran

bin/kafka-server-start.sh config/server.properties 

I have a producer posting to topic "test" because I ran

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test yello is this thing on? let's try another gimme more 

When I run the old API consumer, it works by running

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning 

However, when I run the new API consumer, I don't get anything when I run

bin/kafka-console-consumer.sh --new-consumer --topic test --from-beginning \     --bootstrap-server localhost:9092 

Is it possible to subscribe to a topic from the console consumer using the new api? How can I fix it?

like image 762
EthanP Avatar asked Jan 17 '16 21:01

EthanP


People also ask

How do I receive messages in Kafka topic?

You can use a KafkaConsumer node in a message flow to subscribe to a specified topic on a Kafka server. The KafkaConsumer node then receives messages that are published on the Kafka topic, as input to the message flow.

How do you ensure consumers receive messages in the correct order with Kafka?

How to Ensure the Order of Messages. In Kafka, order can only be guaranteed within a partition. This means that if messages were sent from the producer in a specific order, the broker will write them to a partition and all consumers will read from that in the same order.

What if consumer fails in Kafka?

If the consumer fails after saving the offsets back to Kafka but before writing the data to the database, it will skip these records next time it runs and data will be lost.


1 Answers

I my MAC box I was facing the same issue of console-consumer not consuming any messages when used the command

kafka-console-consumer --bootstrap-server localhost:9095 --from-beginning --topic my-replicated-topic 

But when I tried with

kafka-console-consumer --bootstrap-server localhost:9095 --from-beginning --topic my-replicated-topic --partition 0 

It happily lists the messages sent. Is this a bug in Kafka 1.10.11?

like image 106
Reghuram Vidyarthi Avatar answered Oct 05 '22 09:10

Reghuram Vidyarthi