Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

JSON output format for Hive Query results

Tags:

Is there any way to convert the Hive query result in JSON format?

like image 223
divinedragon Avatar asked Apr 03 '12 14:04

divinedragon


People also ask

How do I export hive query results?

To directly save the file in HDFS, use the below command: hive> insert overwrite directory '/user/cloudera/Sample' row format delimited fields terminated by '\t' stored as textfile select * from table where id >100; This will put the contents in the folder /user/cloudera/Sample in HDFS. Show activity on this post.

What is JSON output format?

JavaScript Object Notation (JSON) is a standard text-based format for representing structured data based on JavaScript object syntax. It is commonly used for transmitting data in web applications (e.g., sending some data from the server to the client, so it can be displayed on a web page, or vice versa).

Does hive support JSON data type?

Hive provides three different mechanisms to run queries on JSON documents, or you can write your own: Use the get_json_object user-defined function (UDF). Use the json_tuple UDF. Use the custom Serializer/Deserializer (SerDe).


2 Answers

This seems to come up quite often. Use the to_json UDFs' from Brickhouse (http://github.com/klout/brickhouse ). If you convert your results to a named_struct, it will interpret it as a JSON map, and output accordingly.

SELECT to_json( named_struct( "field1", field1 ,             "field2", field2,             "field3", field3 ) )    FROM mytable; 

The to_json will also interpret arrays and maps accordingly.

like image 114
Jerome Banks Avatar answered Sep 28 '22 13:09

Jerome Banks


I was using a tool called Apache Nifi. It has AvrotoJSON processor. The hive output which is in Avro format can be easily converted to JSON. The below link will be helpful: https://nifi.apache.org/

like image 44
ForeverLearner Avatar answered Sep 28 '22 14:09

ForeverLearner