kafka-console-producer ignores value serializer?

Question

I have need to change the value serializer/deserializer in my Kafka setup (for testing purposes I have been using the IntegerSerializer/IntegerDeserializer). With the JAVA API, it works exactly as expected; however when using the console tools, it doesn't seem to work correctly.

All of the trouble shooting that I have done has lead me to one conclusion: It appears that the kafka-console-producer is ignoring the Serializer options. I have tried --property value.serializer and --producer-property value.serializer and --value-serializer and setting it in a configuration with the --producer.config argument.

Its not just failing to Serialize the data as an integer, it doesn't even care if I enter nonsense as the argument value. With the kafka-console-consumer (which works perfectly with its IntegerDeserializer) if I enter --value-deserializer not.a.real.class it will throw a class not found exception; however --value-serializer not.a.real.class on the kafka-console-producer, does nothing.

Setting these values on the producer in the JAVA API, works perfectly (and will throw an exception if I enter nonsense as the serializer class) but not when using the kafka-console-producer.

This is an example of one of the ways I tried to run the console producer:

kafka-console-producer --broker-list IPADDRESS:9092 --topic TOPIC --property value.serializer=org.apache.kafka.common.serialization.IntegerSerializer

And the related consumer:

kafka-console-consumer --bootstrap-server IPADDRESS:9092 --topic TOPIC  --property value.deserializer=org.apache.kafka.common.serialization.IntegerDeserializer

Has anyone managed to get this working with the command line tools? Is there something obvious that I am missing?

Answer 1

This is a known bug in the console producer. you can work around it by passing the --old-producer option to the tool.