Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unable to communicate with kafka server using kafka Producer API

I have setted up kafka on a single node and have started zookeeper as well as kafka server.I tested it for internal producer and consumer on console and it works well.But when I am running a internal kafka consumer on console and my custom producer it does not work.

Below is my Producer class

    Properties props = new Properties();

    props.put("metadata.broker.list", "xx.xx.xx.xx:9092");
    props.put("serializer.class", "kafka.serializer.StringEncoder");
    props.put("partitioner.class", "com.example.producer.SimplePartitioner");
    props.put("request.required.acks", "1");

    ProducerConfig config = new ProducerConfig(props);

    Producer<String, String> producer = new Producer<String, String>(config);
    KeyedMessage<String, String> data = new KeyedMessage<String, String>(
            "mails", "xxxx");
    producer.send(data);

When the control reaches the producer.send(), it stops after 3 tries with the following exception

java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.Producer.send(Producer.scala:77)
at kafka.javaapi.producer.Producer.send(Producer.scala:33)
like image 565
Android Mason Avatar asked Mar 17 '15 12:03

Android Mason


Video Answer


1 Answers

I was trying to connect to kafka server from an Producer class in eclipse from external VM. I had to replace localhost with ip address in the producer.properties in config/ of kafka.

like image 58
Android Mason Avatar answered Oct 16 '22 21:10

Android Mason