Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Create Hive table to read parquet files from parquet/avro schema

Tags:

hive

avro

parquet

We are looking for a solution in order to create an external hive table to read data from parquet files according to a parquet/avro schema.

in other way, how to generate a hive table from a parquet/avro schema ?

thanks :)

like image 712
Mehdi TAZI Avatar asked Dec 10 '15 13:12

Mehdi TAZI


1 Answers

Try below using avro schema:

CREATE TABLE avro_test ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS AVRO TBLPROPERTIES ('avro.schema.url'='myHost/myAvroSchema.avsc'); 

CREATE EXTERNAL TABLE parquet_test LIKE avro_test STORED AS PARQUET LOCATION 'hdfs://myParquetFilesPath';

Same query is asked in Dynamically create Hive external table with Avro schema on Parquet Data

like image 103
Ram Manohar Avatar answered Oct 14 '22 23:10

Ram Manohar