Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hive drop table not working - Hanging

Tags:

hive

I tried dropping a HIVE table. Screen hangs. I tried in both HIVE CLI and beeline. Neither the table getting dropped nor the data getting inserted into the table. Need help!

like image 494
TamizhK Avatar asked Dec 29 '16 15:12

TamizhK


People also ask

How do I force drop a table in hive?

In Hive,” user/hive/warehouse” is the default directory. Internal tables are stored in this directory by default. We do not have to provide the location manually while creating the table. “Drop table” command deletes the data permanently.

What happens when external table is dropped?

Dropping an external table removes all table-related metadata. It doesn't delete the external data.

Does drop table delete data hive?

DROP TABLE command in the hive is used to drop a table inside the hive. Hive will remove all of its data and metadata from the hive meta-store. The hive DROP TABLE statement comes with a PURGE option.

How do I know if my hive table is locked?

log in /tmp/$user and you can see that the table is being Locked. “Show Locks” Command in the Hive CLI. It will display the Locks for all the tables in the database. Please check for the table's name against the Show Locks Command's output.

How do I drop a table in hive?

Hive is very much capable such that it can query petabytes of records stored inside the hive table. DROP TABLE command in the hive is used to drop a table inside the hive. Hive will remove all of its data and metadata from the hive meta-store. The hive DROP TABLE statement comes with a PURGE option.

Why can't I open a table in hive?

This is usually caused by the table being an external table that doesn't allow Hive to perform all operations on it. Another thing you can try is what's suggested in this thread (i.e. before you drop the table, change its property to be EXTERNAL=FALSE).

Where are internal tables stored in hive?

Internal tables are stored in this directory by default. We do not have to provide the location manually while creating the table. “Drop table” command deletes the data permanently. Hive manages all the security for managed tables. I do have a table already present in the “user/hive/warehouse” directory called “codes”.

What is drop data in hive?

The keyword “DROP” refers to deletion. For the deletion of data, we require data to be present in the hive. For data storage, HDFS (Hadoop Distributed File System) comes into the picture. Now when we say we have data in hive table it means two things:


2 Answers

Actually the table was locked with some queries. After unlocking the table, I am able to drop the table now.

=>Query to unlock the table

unlock table <tablename>

=>We can find the tables which are locked in the Hive CLI /Beeline:

show locks extended;

like image 88
TamizhK Avatar answered Oct 30 '22 22:10

TamizhK


I had simular issue but I was not able even to drop hive database which had this table inside.

I had to manually delete from metastoredb coruppted record.

Before do that of course do mysqldump just in case :)

In table TBLS record which was coresponding to the table. Record atributes SD_ID, VIEW_EXPANDED_TEXT,VIEW_ORIGINAL_TEXT were null.

To find it:

SELECT DB.DB_ID,
       DB.NAME AS DB_NAME,
       TB.SD_ID,
       TB.TBL_ID,
       TB.TBL_NAME,
       SD.SD_ID,
       SD.INPUT_FORMAT,
       CD.*
FROM TBLS AS TB
     left JOIN DBS AS DB ON(DB.DB_ID = TB.DB_ID)
     left JOIN SDS AS SD ON(TB.SD_ID = SD.SD_ID)
     left JOIN COLUMNS_V2 AS CD ON(SD.CD_ID = CD.CD_ID)
WHERE 
TB.TBL_NAME='your_name';

Then I had to delete all FK of this record inside two other tables:

 delete from TBL_COL_PRIVS where TBL_ID=xxxx ;
 delete from TBL_PRIVS where TBL_ID=xxxx ;
 delete from TBLS where TBL_ID=xxxx ;
like image 31
Paco Avatar answered Oct 30 '22 22:10

Paco