Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does DROP PARTITION delete data from external table in HIVE?

An external table in HIVE is partitioned on year, month and day.

So does the following query delete data from external table for the specific partitioned referenced in this query?:-

ALTER TABLE MyTable DROP IF EXISTS PARTITION(year=2016,month=7,day=11);
like image 826
Dhiraj Avatar asked Jul 12 '16 01:07

Dhiraj


People also ask

Does drop partition delete data?

Data itself are stored in files in the partition location(folder). If you drop partition of external table, the location remain untouched, but unmounted as partition (metadata about this partition is deleted).

Does dropping an external table delete the data?

Dropping an external table removes all table-related metadata. It doesn't delete the external data.

How do I delete data from an external table in Hive?

Run DROP TABLE on the external table. DROP TABLE names_text; The table is removed from Hive Metastore and the data stored externally. For example, names_text is removed from the Hive Metastore and the CSV file that stored the data is also deleted from HDFS.


1 Answers

Partitioning scheme is not data. Partitioning scheme is part of table DDL stored in metadata (simply saying: partition key value + location where the data-files are being stored).

Data itself are stored in files in the partition location(folder). If you drop partition of external table, the location remain untouched, but unmounted as partition (metadata about this partition is deleted). You can have few versions of partition location unmounted (for example previous versions).

You can drop partition and mount another location as partition (alter table add partition) or change existing partition location. Also drop external table do not delete table/partitions folders with files in it. And later you can create table on top of this location.

Have a look at this answer for better understanding external table/partition concept: It is possible to create many tables (both managed and external at the same time) on top of the same location in HDFS.

like image 165
leftjoin Avatar answered Sep 21 '22 05:09

leftjoin