Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Add PARTITION after creating TABLE in hive

i have created a non partitioned table and load data into the table,now i want to add a PARTITION on the basis of department into that table,can I do this? If I do:

ALTER TABLE Student ADD PARTITION (dept='CSE') location '/test';

It gives me error:

FAILED: SemanticException table is not partitioned but partition spec exists: {dept=CSE}

please help. Thanks

like image 355
rinku buragohain Avatar asked Jun 18 '15 06:06

rinku buragohain


People also ask

Can we apply the partitioning on the already existing Hive table?

Make sure no other process is writing to the table. Create new external table using partitioning. Insert into new table by selecting from the old table. Drop the new table (external), only table will be dropped but data will be there.

How do I add a partition to an existing table?

Use the ALTER TABLE ADD PARTITION statement to add a new partition to the "high" end (the point after the last existing partition). To add a partition at the beginning or in the middle of a table, use the SPLIT PARTITION clause.

Can we add partitions in existing non partitioned table in Hive?

Unfortunately, you cannot add/create partition in existing table which was not partitioned while creation of the table.

How do I update my Hive partition?

You can refresh Hive metastore partition information manually or automatically. You run the MSCK (metastore consistency check) Hive command: MSCK REPAIR TABLE table_name SYNC PARTITIONS every time you need to synchronize a partition with your file system. You set up partition discovery to occur periodically.


2 Answers

First create a table in such a way so that you don't have partition column in the table.

create external table Student(col1 string, col2 string) partitioned by (dept string) location 'ANY_RANDOM_LOCATION';

Once you are done with the creation of the table then alter the table to add the partition department wise like this :

alter table Student add partition(dept ='cse') location '/test';

I hope this will help.

like image 165
Naresh Avatar answered Oct 11 '22 12:10

Naresh


You can't alter table partition if you didn't define partition while creation of table.

If, when altering a un-partitioned table to add partition, you get this error: "Semantic Exception table is not partitioned but partition spec exists: {dept=CSE}," it means you are trying to include the partitioned in the table itself.

You are not getting syntax error because the syntax of the command is correct and used to alter the partition column.

Learn more about Hive Tables:

https://www.dezyre.com//hadoop-tutorial/apache-hive-tutorial-tables

You can also check the possible alternation in table:

https://sites.google.com/site/hadoopandhive/home/how-to-create-table-partition-in-hive

Hope this helps.

like image 33
Abhijit Kumar Avatar answered Oct 11 '22 13:10

Abhijit Kumar