Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hive- how do I "create table as select.." with partitions from original table?

I need to create a "work table" from our hive dlk. While I can use:

create table my_table as
select *
from dlk.big_table

just fine, I have problem with carrying over partitions (attributes day, month and year) from original "big_table" or just creating new ones from these attributes. Searching the web did not really helped me answer this question- all "tutorials" or solutions deal either with create as select OR creating partitions, never both. Can anybody here please help?

like image 958
MageInTraining Avatar asked May 06 '19 07:05

MageInTraining


People also ask

Can we apply the partitioning on the already existing Hive table?

Unfortunately, you cannot add/create partition in existing table which was not partitioned while creation of the table.

How will you create a table from another table without data in Hive?

Copy the table structure in Hive. You want to create the new table from another table. But you don't want to copy the data from the old table to new table. In that case, We can use Create table Like option in Hive.


1 Answers

Creating partitioned table as select is not supported. You can do it in two steps:

  1. create table my_table like dlk.big_table; This will create table with the same schema.

  2. Load data.

    set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict;

    insert overwrite table my_table partition (day, month, year) select * from dlk.big_table;

like image 188
leftjoin Avatar answered Oct 12 '22 11:10

leftjoin