Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Nested specific type de-serialization with Avro

Here is the related part of the schema that I am using to generate my Avro class

{"namespace": "com.namespace.kafka.event",
 "type": "record",
 "name": "RecordName",
 "fields": [

    // some fields declared
    ...

    {"name": "products", "type": { "type": "array", "items": 
        {
            "name": "productInfo",
            "type": "record",
            "fields" : [
                {"name": "productId", "type": "int"},
                {"name": "productScore", "type": "float"}
            ]
        }
    }
},

// some more fields declared
...

] }

I am able to generate classes for this schema and to populate them. So basically in the generation a new class called productInfo is generated which serves my needs. (Later I send the data to Kafka). The problem is the de-serialization. I am using something like this:

GenericData.Array<String> productTuple = (GenericData.Array<String>) record.get("products");

This returns me a string (which is actually a JSON) like this for each element of GenericData array.

{"productName" : someName, "productScore" : someScore }

I can parse this string and get what I want but is there a more "Avro" way of doing this. Like can I cast the results again to a GenericRecord object like this.

GenericData.Array<GenericRecord> productTuple = (GenericData.Array<GenericRecord>) record.get("products");

or will this work where productInfo being the class generated from my avro schema:

GenericData.Array<productInfo> productTuple = (GenericData.Array<productInfo>) 

Unfortunately, I cant test it I dont have an easy setup to do so. I searched the whole web but could not find a way to do that. Any help will be much appreciated.

like image 592
ralzaul Avatar asked Mar 17 '23 18:03

ralzaul


1 Answers

The right way to do it was to use :

GenericData.Array<GenericRecord> productTuple =(GenericData.Array<GenericRecord>) record.get("products");

I believe the last one might work as well but this one lets me deserialize the JSON without any further interference from a class.

like image 105
ralzaul Avatar answered Mar 25 '23 06:03

ralzaul