Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache Avro for c# vs Microsoft.Avro.Core

I am new to avro and am trying to implement avro schema serialization in a new project. I saw that there are two different NuGet packages available and was wondering what the differences between both were.

  1. Microsoft's version: https://github.com/welly87/Apache-Avro-Core (couldnt find any documentation for this except for the hadoop and azure version)
  2. Apache's version 1.7.7.2 :https://avro.apache.org/docs/1.7.7/api/csharp/index.html

Also are there any available examples to look at how serialization is done or best practices for avro serialization in c# given any generic schema. I see a lot of info for java but barely any for c#. Which of the two is it the wisest to use?

like image 623
afkjlkdo7635 Avatar asked Oct 17 '17 22:10

afkjlkdo7635


People also ask

When should I use Apache Avro?

Apache Avro is especially useful while dealing with big data. It offers data serialization in binary as well as JSON format which can be used as per the use case. The Avro serialization process is faster, and it's space efficient as well.

What does Apache Avro stand for?

Apache Avro is a data serialization and remote procedure call framework which is developed within the Apache Hadoop project where it provides both a serialization format to get persistent data and a wire format for providing communication between Hadoop nodes, as well as connecting client programs to the Hadoop ...

Is Avro same as JSON?

Avro has a JSON like data model, but can be represented as either JSON or in a compact binary form. It comes with a very sophisticated schema description language that describes data. We think Avro is the best choice for a number of reasons: It has a direct mapping to and from JSON.


1 Answers

It appears the the Microsoft.Avro.Core package is no longer in pre-release and you can find the source at the microsoft-avro repository.

There is currently not a lot of documentation, but there are quite a few unit tests you can glean usage from.

Specifically, take a look at the AvroContainerTests, there are tests that show reading/writing using both the Microsoft types and Apache types.

The original announcement stated "The library can generate portable files with embedded schema. The file format is compatible with Avro container file specification and can be used across platforms", so for embedded schema you will want to use the AvroContainer.

There is also the Official Sample Test you can use as a starting point.

like image 192
Oppositional Avatar answered Oct 04 '22 15:10

Oppositional