the parquet docs from cloudera shows examples of integration with pig/hive/impala. but in many cases I want to read the parquet file itself for debugging purposes.
is there a straightforward java reader api to read a parquet file ?
Thanks Yang
This project provides a library that reads Parquet files into Java objects.
With the query results stored in a DataFrame, we can use petl to extract, transform, and load the Parquet data. In this example, we extract Parquet data, sort the data by the Column1 column, and load the data into a CSV file.
parquet file formats. You can open a file by selecting from file picker, dragging on the app or double-clicking a . parquet file on disk. This utility is free forever and needs you feedback to continue improving.
Old method: (deprecated)
AvroParquetReader<GenericRecord> reader = new AvroParquetReader<GenericRecord>(file);
GenericRecord nextRecord = reader.read();
New method:
ParquetReader<GenericRecord> reader = AvroParquetReader.<GenericRecord>builder(file).build();
GenericRecord nextRecord = reader.read();
I got this from here and have used this in my test cases successfully.
You can use AvroParquetReader
from parquet-avro library to read a parquet file as a set of AVRO GenericRecord
objects.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With