Kafka AVRO - conversion from long to datetime

Question

I get the following error when I want to send an AVRO message which contains a field that has the type long:

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 61
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to org.joda.time.DateTime

I use Confluent 3.2.0 and Apache Spark 2.2.0. This error is thrown in a Spark Job which processes AVRO messages and prints them in a console. In the AVRO schema, the corresponding field is defined like this:

{\"name\": \"event_time\", \"type\": { \"type\" : \"long\", \"logicalType\": \"timestamp-millis\"}}

In the Java class generated from the .avsc file, the field is defined as below:

private DateTime event_time;

Answer 1

I encountered a similar issue using Confluent 4.0.0 and Avro 1.8.2. I had a stream processor that was attempting to convert a long to a DateTime. I overcame the issue by adding the correct conversion. Before I begin any processing logic, I used the Specific Data static utilities class and adding the correct Logical type conversion.

SpecificData.get().addLogicalTypeConversion(new TimeConversions.TimestampConversion());

Answer 2

In Avro 1.9.X and above, the symbol TimestampConversion is not present anymore. replacing the above from @user3222582 with TimestampMillisConversion fixed a compilation error for me with avro 1.9.2