Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do you insert data into complex data type "Struct" in Hive

I'm completely new to Hive and Stack Overflow. I'm trying to create a table with complex data type "STRUCT" and then populate it using INSERT INTO TABLE in Hive.

I'm using the following code:

CREATE TABLE struct_test
(
 address STRUCT<
                houseno:    STRING
               ,streetname: STRING
               ,town:       STRING
               ,postcode:   STRING
               >
);

INSERT INTO TABLE struct_test
SELECT NAMED_STRUCT('123', 'GoldStreet', London', W1a9JF') AS address
FROM dummy_table
LIMIT 1;

I get the following error:

Error while compiling statement: FAILED: semanticException [Error 10044]: Cannot insert into target because column number type are different 'struct_test': Cannot convert column 0 from struct to array>.

I was able to use similar code with success to create and populate a data type Array but am having difficulty with Struct. I've tried lots of code examples I've found online but none of them seem to work for me... I would really appreciate some help on this as I've been stuck on it for quite a while now! Thanks.

like image 532
data101 Avatar asked Sep 08 '16 15:09

data101


People also ask

Does Hive support complex data types?

In addition to primitive data types, Hive also supports a few complex data types: Struct , MAP , and Array . Complex data types are also known as collection data types.

How do you access struct data type in Hive?

Struct – a complex data type in Hive which can store a set of fields of different data types. The elements of a struct are accessed using dot notation.


2 Answers

your sql error. you should use sql:

INSERT INTO TABLE struct_test 
       SELECT NAMED_STRUCT('houseno','123','streetname','GoldStreet', 'town','London', 'postcode','W1a9JF') AS address 
           FROM dummy_table LIMIT 1;
like image 97
aaronshan Avatar answered Sep 23 '22 08:09

aaronshan


You can not insert complex data type directly in Hive.For inserting structs you have function named_struct. You need to create a dummy table with data that you want to be inserted in Structs column of desired table. Like in your case create a dummy table

CREATE TABLE DUMMY ( houseno:    STRING
           ,streetname: STRING
           ,town:       STRING
           ,postcode:   STRING);

Then to insert in desired table do

INSERT INTO struct_test SELECT named_struct('houseno',houseno,'streetname'
                  ,streetname,'town',town,'postcode',postcode) from dummy;
like image 24
Amit_Hora Avatar answered Sep 25 '22 08:09

Amit_Hora