I know that there is
DESCRIBE FORMATTED table_name;
that shows you the table format. Is there a way to get a more information about partitions apart from
SHOW PARTITIONS table_name;
I saw that Hive language manual has this
DESCRIBE [EXTENDED|FORMATTED] [db_name.]table_name PARTITION partition_spec
I would like to view all the partitions along with the url in hdfs or s3 where the data is stored.
Show All Partitions on Hive Table. After loading the data into the Hive partition table, you can use SHOW PARTITIONS command to see all partitions that are present. Alternatively, if you know the Hive store location on the HDFS for your table, you can run the HDFS command to check the partitions.
The HIVE_DEFAULT_PARTITION in hive is represented by a NULL value of the partitioned column. That means, if we have a NULL value for a partition column and loading this record to a partitioned table, then hive_default_partition will get create for that record.
Static Partitioning in Hive While loading data, you need to specify which partition to store the data in. This means that with each load, you need to specify the partition column value. You can add a partition in the table and move the data file into the partition of the table.
All tables have at least one partition, so if you are looking specifically for partitioned tables, then you'll have to filter this query based off of sys. partitions. partition_number <> 1 (for non-partitioned tables, the partition_number is always equal to 1).
To show partitions:
show partitions table_name
To show where a partition is physically stored:
describe formatted dbname.tablename partition (name=value)
I don't know of a built-in way to create an output that is (partition, path) but you can build it using these two commands and some grep/awk or whatever.
analyze table TABLENAME partition(the_partition) compute statistics nopass;
The above code gives you more info about the partitions (number of files, number of rows, total size), but doesn't give you exact location.
If you want exact location, you may want to create an external table.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With