Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to deserialize avro in nodejs?

Tags:

node.js

avro

I have an avro file. I want to use nodejs to open and read its schema and iterate through its records. How to do this? The avro libraries I see in nodejs appear to require you to pass in a schema instead of getting the schema out of the .avro file. Also, I want to be able to support arrays, which there does not seem to exist a node library that does (node-avro-io).

My avro/avroschema Contains:

  1. A nested field {a:{suba: vala, subb: vala}}.
  2. An array field {a:["A","B"]}. node-avro-io does not work.

Error I get with node-avro-io:

Avro Invalid Schema Error: Primitive type must be one of: ["null","boolean","int","long","float","double","bytes","string"]; got DependencyNode
like image 582
Rolando Avatar asked Feb 23 '26 05:02

Rolando


1 Answers

In case you're still looking, you can do this with avsc. The code would look something like:

var avro = require('avsc');

// To stream the file's records (they will already be decoded):
avro.createFileDecoder('your/data.avro')
  .on('data', function (record) { /* Do something with the record. */ });

// Or, if you just want the file's header (which includes the schema):
var header = avro.extractFileHeader('your/data.avro');
like image 90
mtth Avatar answered Feb 25 '26 21:02

mtth



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!