Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

UnknownHostException kafka

I am trying to setup a Kafka cluster (the first node in the cluster actually).

I have a single node zookeeper cluster setup. I am setting up kafka on a separate node.

Both running CentOS 6.4, running IPV6 which is a bit of a PITA. I verified that the machines can talk to each other using netcat.

When I startup kafka, I am getting the following exception (which causes kafka to shut down). EDIT: I got kafka starting, I had to set the host.name property in the server.config file.

I was able to create a test topic and send messages just fine from the kafka server.

However, I get the same error when trying to consume the messages.

Any help, suggestions?

bin/kafka-console-consumer.sh --zookeeper zk1:2181 --topic test --from-beginning
Exception in thread "main" java.net.UnknownHostException: kafka: kafka: Name or service not known
    at java.net.InetAddress.getLocalHost(InetAddress.java:1473)
    at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:107)
    at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:128)
    at kafka.consumer.Consumer$.create(ConsumerConnector.scala:89)
    at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:178)
    at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
Caused by: java.net.UnknownHostException: kafka: Name or service not known
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
    at java.net.InetAddress.getLocalHost(InetAddress.java:1469)
    ... 5 more
like image 622
noplay Avatar asked Aug 26 '14 02:08

noplay


People also ask

What can cause UnknownHostException?

UnknownHostException is a common error message in Java applications. This error typically indicates that there was a DNS resolution failure. If a Java application fails to get a valid DNS answer, then it might throw an UnknownHostException error.

How do I resolve unknown host exception in spring boot?

A few tips to prevent the exception are: Double-check the hostname: Make sure there is no typo, and trim all whitespaces. Check the system's DNS settings: Make sure the DNS server is up and reachable, and if the hostname is new, wait for the DNS server to catch up.

What is advertised listeners in Kafka?

Advertised listeners is the most important setting of Kafka and setting it correctly ensures your clients all over your network can successfully connect to every broker in your Kafka cluster.


1 Answers

When you run > bin/kafka-console-consumer.sh command kafka loads a ConsoleConsumer, which will attempt to create a consumer with an auto generated consumer id. The way Kafka generates the consumer id is to concatenate the name of the local host to it. So, in the problem was the fact that java could not resolve the ip address for local host on the Open Stack VM I am working with.

So the answer was that the Open Stack VM was resolving the local host name to kafka, which is the name of the VM. I had everything setup in the Kafka and Zookeeper instances as kafka1.

So, when java was calling getLocalHost, it was trying to find the IP Address for kafka, which I did not have in my /etc/hosts file.

I simply added an entry for kafka in my /etc/hosts file and everything started working wonderfully!!!

I would have thought it would resolve to localhost, but it did not, it resolved to the name of the vm, kafka.

like image 74
noplay Avatar answered Sep 27 '22 22:09

noplay