Menu
I'm getting this error from AWS Athena:
HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'id' in table 'db.app_events' is declared as type 'string', but partition 'xxxxx' declared column 'data.entity_price' as type 'double'.
there is no connection with those fields but it gives an error. Is it a bug or am I missing something?
353
Open the Athena console at https://console.aws.amazon.com/athena/ . In the query editor, next to Tables and views, choose Create, and then choose AWS Glue crawler. Follow the steps on the Add crawler page of the AWS Glue console to add a crawler. For more information, see Using AWS Glue crawlers.
What are the Hive Partitions? Apache Hive organizes tables into partitions. Partitioning is a way of dividing a table into related parts based on the values of particular columns like date, city, and department. Each table in the hive can have one or more partition keys to identify a particular partition.
You can by using the Athena JDBC driver. This approach circumvents the catalog, as only Athena (and not Glue as of 25-Jan-2019) can directly access views. Download the driver and store the jar to an S3 bucket. Specify the S3 path to the driver as a dependent jar in your job definition.
To see the change in table columns in the Athena Query Editor navigation pane after you run ALTER TABLE REPLACE COLUMNS , you might have to manually refresh the table list in the editor, and then expand the table again. ALTER TABLE REPLACE COLUMNS does not work for columns with the date datatype.
Schema is kept at partition level unless you inherit schema from table. See this: https://docs.aws.amazon.com/athena/latest/ug/updates-and-partitions.html
also discussed here as well: How to solve this HIVE_PARTITION_SCHEMA_MISMATCH?
173
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
