Unable to list kafka topics in openwhisk setup

Question

Setup details: I am setting up openwhisk on my local ubuntu(16.04) vm. in this setup kafka is running in one docker and zookeeper in another docker.

I connect to the the kafka docker using cmd

sudo docker exec -it <container id> sh

once connected i execute the following command to get the list of topics

bin/kafka-topics.sh --list --zookeeper localhost:2181

which gives me an exception

Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 7203; nested exception is:
        java.net.BindException: Address already in use

i am unable to understand why is it trying to use 7203 port?

docker ps output

83eba3961247        ches/kafka:0.10.0.1              "/start.sh"              
11 days ago         Up 23 hours         7203/tcp, 0.0.0.0:9092->9092/tcp                                                                                                                       
kafka
947fa689a7ef        zookeeper:3.4                    "/docker-
entrypoin..."   11 days ago         Up 23 hours         2888/tcp, 
0.0.0.0:2181->2181/tcp, 3888/tcp                                                                                                             zookeeper

Answer 1

The Kafka container OpenWhisk is using sets a JMX_PORT by default. That's the 7203 port you're seeing. To get your script to work you need to unset that environment setting:

unset JMX_PORT; bin/kafka-topics.sh --list --zookeeper localhost:2181

Note though, that localhost is not a valid address for your zookeeper instance, as it refers to the localhost of the current container, which is not Zookeeper. If you exchange localhost with the external IP of your VM or the IP of the zookeeper container (get it via docker inspect zookeeper --format {{.NetworkSettings.Networks.bridge.IPAddress}}) your topics should be listed fine.