I'm querying a table in Athena that is giving the error: GENERIC_INTERNAL_ERROR: Number of partition values does not match number of filters
I was able to query it earlier, but added another partition (AWS glue job) to try and optimize joins I will be doing in the query later. I did re-run the crawler to update the table to reflect the partitions.
I'm wondering what this error actually means.
I tried recrawling the data, but no luck. Next up I'm wiping out the underlying S3 data and going to reprocess the entire set from scratch and then re-crawl it.
SELECT *
FROM mydb.mytable
LIMIT 10
I'd like to understand what the error means and if there is a way around it other than reprocessing the entire raw data set and re-crawling it.
Although Athena supports querying AWS Glue tables that have 10 million partitions, Athena cannot read more than 1 million partitions in a single scan.
Incorrect LOCATION path If the input LOCATION path is incorrect, then Athena returns zero records.
I had the same issue today. In my case, it was because some of the underlying data in s3 had been removed, so the number of partitions in the Glue table didn't match the number of files in s3.
To fix it, I just deleted the tables in Glue, then re-ran the crawler.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With