Kafka + Zookeeper: Connection to node -1 could not be established. Broker may not be available

Question

UPD: if you are running in single-node mode:

I have seen this message in spark console log while trying to deploy application. Solved by changing this parameter in server.properties:

listeners=PLAINTEXT://myhostname:9092

to

listeners=PLAINTEXT://localhost:9092

make sure that you have java process listening on 9092 with netstat -lptu

Change:

#listeners=PLAINTEXT://:9092`

in server.properties to:

listeners=PLAINTEXT://localhost:9092

Note: You also need to uncomment this statement aka remove the # symbol.

I found the error. Observing zookeeper logs when the server started I noticed:

server.1=mylocal-0.:2888:3888

with a dot (.) after the name of the host.

The script that produces the zookeeper's config is from https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/zkGenConfig.sh

Looking inside I see that DOMAIN is not filled:

HOST=`hostname -s`
DOMAIN=`hostname -d`

function print_servers() {
    for (( i=1; i<=$ZK_REPLICAS; i++ ))
    do
        echo "server.$i=$NAME-$((i-1)).$DOMAIN:$ZK_SERVER_PORT:$ZK_ELECTION_PORT"
    done
}

For my case (localhost) I don't need domain, so I removed that variable.

Now zookeeper and kafka communicate with no errors.

If you want to set up it for local then you need to un comment the below line in path_to_kafka_folder\kafka_2.13-2.6.0\config\server.properties

listeners=PLAINTEXT://localhost:9092