Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hive: how to show all partitions of a table?

Tags:

hadoop

hive

I have a table with 1000+ partitions.

"Show partitions" command only lists a small number of partitions.

How can i show all partitions?

Update:

  1. I found "show partitions" command only lists exactly 500 partitions.

  2. "select ... where ..." only processes the 500 partitions!

like image 903
Kevin Leo Avatar asked Mar 25 '13 13:03

Kevin Leo


People also ask

How can I see all partitions in Hive table?

You can see Hive MetaStore tables,Partitions information in table of "PARTITIONS". You could use "TBLS" join "Partition" to query special table partitions. Show activity on this post.

How do I know how many partitions Hive?

Say if your hive table size was aprrox 14.8 GB then it will divide the hive table data into 128 MB blocks and will result in 119 Partitions. On the other hand your hive table is partitioned so the partition column has 150 unique values.

How do I get a list of all the partitioned tables in my database?

The sys. partitions catalog view gives a list of all partitions for tables and most indexes. Just JOIN that with sys. tables to get the tables.

How many partitions can a Hive table have?

Current Hive versions with RDBMS metastore backend should be able to handle 10000+ partitions.


2 Answers

CLI has some limit when ouput is displayed. I suggest to export output into local file:

$hive -e 'show partitions table;' > partitions 
like image 173
www Avatar answered Sep 21 '22 13:09

www


hive> show partitions table_name;

like image 44
Sri Avatar answered Sep 17 '22 13:09

Sri