Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Presto and hive partition discovery

I'm using presto mainly with hive connector to connect to hive metastore.

All of my tables are external tables pointing to data stored in S3.

My main issue with this is that there is no way (at least on I'm aware of ) to do partition discovery in Presto ,so before I start query a table in presto I need to switch to hive and run msck repair table mytable

is there more reasonable way to do it in Presto?

like image 469
Lior Baber Avatar asked Dec 27 '15 06:12

Lior Baber


2 Answers

I'm on version 0.227 and the following helps me:

select * from hive.yourschema."yourtable$partitions"

This select returns all the partitions mapped in your catalog. You can filter, order, etc. as a normal query would.

like image 73
Ricardo Pedrotti Avatar answered Oct 07 '22 07:10

Ricardo Pedrotti


No.

If the HIVE metastore doesn't see the partitions, PrestoDB will not see it.

Maybe a cron can help you.

like image 27
Damien Carol Avatar answered Oct 07 '22 08:10

Damien Carol