getting "org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1195725856 larger than 104857600)"

Question

I have installed zookeeper and kafka, first step : running zookeeper by the following commands :

bin/zkServer.sh start bin/zkCli.sh 

second step : running kafka server

bin/kafka-server-start.sh config/server.properties 

kafka should run at localhost:9092

but I am getting the following error :

WARN Unexpected error from /0:0:0:0:0:0:0:1; closing connection (org.apache.kafka.common.network.Selector) org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1195725856 larger than 104857600) 

I am following the following link : Link1 Link2

I am new to kafka ,please help me to set it up.

Answer 1

1195725856 is GET[space] encoded as a big-endian, four-byte integer (see here for more information on how that works). This indicates that HTTP traffic is being sent to Kafka port 9092, but Kafka doesn't accept HTTP traffic, it only accepts its own protocol (which takes the first four bytes as the receive size, hence the error).

Since the error is received on startup, it is likely benign and may indicate a scanning service or similar on your network scanning ports with protocols that Kafka doesn't understand.

In order to find the cause, you can find where the HTTP traffic is coming from using tcpdump:

tcpdump -i any -w trap.pcap dst port 9092 # ...wait for logs to appear again, then ^C... tcpdump -qX -r trap.pcap | less +/HEAD 

Overall though, this is probably annoying but harmless. At least Kafka isn't actually allocating/dirtying the memory. :-)

Answer 2

Try to reset socket.request.max.bytes value in $KAFKA_HOME/config/server.properties file to more than your packet size and restart kafka server.