Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Creating a Proper avro schema for timestamp record

I would like to know what the proper avro schema would be for some json to avro conversion that is in this format:

{"entryDate": "2018-01-26T12:00:40.930"}

My schema:

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : "long",
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}

I keep getting

`'Cannot convert field entryDate: Cannot resolve union: 
"2018-01-26T12:00:40.930" 
not in 
["null",{"type":"long","logicalType":"timestamp-millis"}]'`
like image 569
koala421 Avatar asked Jan 26 '18 17:01

koala421


1 Answers

It was a silly mistake...obviously I was storing the timestamp value as a string so the avro schema needed a string instead of long for type.

ie.

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : `**"long"**`,
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}

should be

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : `**"string"**`,
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}

doh!

like image 149
koala421 Avatar answered Sep 17 '22 01:09

koala421