Can i point multiple location to same hive external table?

Question

I need to process the multiple months data simultaneously. So, is there an option to point multiple folders to external table? e.g. Create external table logdata(col1 string, col2 string........) location s3://logdata/april, s3://logdata/march

Answer 1

Simple answer: no, the location of a Hive external table during creation has to be unique, this is needed by the metastore to understand where your table lives.

That being said, you can probably get away with using partitions: you can specify a location for each of your partitions which seems to be what you want ultimately since you are splitting by month.

So create your table like this:

create external table logdata(col1 string, col2 string) partitioned by (month string) location 's3://logdata'

Then you can add partitions like this:

alter table logdata add partition(month='april') location 's3://logdata/april'

You do this for every month, and now you can query your table specifying whichever partition you want, and Hive will only look at the directories for which you actually want data (for example if you're only processing april and june, Hive will not load may)