CSV to AVRO conversion in Azure

Question

I am trying to convert csv files stored in azure data lake store into avro files with created scheme. Is there any kind of example source code which has same purpose?

Answer 1

You can use Azure Data Lake Analytics for this. There is a sample Avro extractor at https://github.com/Azure/usql/blob/master/Examples/DataFormats/Microsoft.Analytics.Samples.Formats/Avro/AvroExtractor.cs. You can easily adapt the code into an outputter.

Another possibility is to fire up an HDInsight cluster on top of your data lake store and use Pig, Hive or Spark.