Menu
Hive Msck repair command is used to repair partitions, but what is full form of MSCK.
I already tried to find in hive doc's but hard luck.
215
The MSCK REPAIR TABLE command scans a file system such as Amazon S3 for Hive compatible partitions that were added to the file system after the table was created. MSCK REPAIR TABLE compares the partitions in the table metadata and the partitions in S3.
MSCK is slow because it is waiting for S3 listing among other things. Athena would read the entire 9.7MB table if it were in one file faster than it would be able to list that huge directory structure. I recommend removing the partitions completely, or if you really must have them then remove p2, p3 and p4 levels.
Similar to how fsckstands for filesystem consistency check, msck is Hive's metastore consistency check.
53
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
