Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Access AWS athena through JPA spring boot

I am trying to use AWS athena using spring boot jpa datasource . I tried setting up datasource with given properties.

    spring.datasource.driver-class-name=com.amazonaws.athena.jdbc.AthenaDriver
    spring.datasource.url=jdbc:awsathena://athena.us-east-1.amazonaws.com:443/default
    spring.datasource.username=*****
    spring.datasource.password=***
    spring.datasource.tomcat.connectionProperties=s3_staging_dir=*****

I am getting below exception

    Caused by:org.springframework.beans.BeanInstantiationException:    Failed to instantiate [org.springframework.orm.jpa.JpaVendorAdapter]: Factory method 'jpaVendorAdapter' threw exception; nested exception is java.lang.IllegalArgumentException: URL must start with 'jdbc'
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189) ~[spring-beans-4.3.7.RELEASE.jar:4.3.7.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(Constructo`enter code here`rResolver.java:588) ~[spring-beans-4.3.7.RELEASE.jar:4.3.7.RELEASE]
... 51 common frames omitted
    Caused by: java.lang.IllegalArgumentException: URL must start with 'jdbc'
at org.springframework.util.Assert.isTrue(Assert.java:92) ~[spring-core-4.3.7.RELEASE.jar:4.3.7.RELEASE]
at org.springframework.boot.jdbc.DatabaseDriver.fromJdbcUrl(DatabaseDriver.java:268) ~[spring-boot-1.5.2.RELEASE.jar:1.5.2.RELEASE]
at org.springframework.boot.autoconfigure.orm.jpa.DatabaseLookup.getDatabase(DatabaseLookup.java:73) ~[spring-boot-autoconfigure-1.5.2.RELEASE.jar:1.5.2.RELEASE]
at org.springframework.boot.autoconfigure.orm.jpa.JpaProperties.determineDatabase(JpaProperties.java:139) ~[spring-boot-autoconfigure-1.5.2.RELEASE.jar:1.5.2.RELEASE]
at org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration.jpaVendorAdapter(JpaBaseConfiguration.java:105) ~[spring-boot-autoconfigure-1.5.2.RELEASE.jar:1.5.2.RELEASE]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration$$EnhancerBySpringCGLIB$$720f8624.CGLIB$jpaVendorAdapter$4(<generated>) ~[spring-boot-autoconfigure-1.5.2.RELEASE.jar:1.5.2.RELEASE]
at org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration$$EnhancerBySpringCGLIB$$720f8624$$FastClassBySpringCGLIB$$9766cf.invoke(<generated>) ~[spring-boot-autoconfigure-1.5.2.RELEASE.jar:1.5.2.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-4.3.7.RELEASE.jar:4.3.7.RELEASE] 
like image 976
Saurabh Sharma Avatar asked Mar 29 '17 09:03

Saurabh Sharma


People also ask

How do I access Athena database?

You can access Athena using the AWS Management Console, a JDBC or ODBC connection, the Athena API, the Athena CLI, the AWS SDK, or AWS Tools for Windows PowerShell.

How does Athena connect to data lake?

Use the Amazon Athena console to query the data in your data lake. Open the Athena console at https://console.aws.amazon.com/athena/ , and sign in as the data analyst, user datalake_user . If necessary, choose Get Started to continue to the Athena query editor. For Data source, choose AwsDataCatalog.

Can I query RDS with Athena?

Configure RDS as Data Source. You configure PostgreSQL RDS instance as the data source for Amazon Athena so that you can query RDS data from the Athena Query Editor. Goto Athena Management console and click on Data sources link.

Can Athena query CSV?

CSV is the only output format used by the Athena SELECT query, but you can use UNLOAD to write the output of a SELECT query to the formats that UNLOAD supports. Although you can use the CTAS statement to output data in formats other than CSV, those statements also require the creation of a table in Athena.


1 Answers

By default, Spring boot will automatically detect which Hibernate Dialect to use based on the Datasource 's metadata retrieved from the JDBC driver.

This error is due to Athena JDBC driver fails to get the related metadata. Not sure why it fails but you can by-pass this detection and explicitly declare which Hibernate Dialect to use by yourself by declaring a JpaVendorAdapter bean :

@Bean
public JpaVendorAdapter jpaVendorAdapter(JpaProperties properties) {
    AbstractJpaVendorAdapter adapter = new HibernateJpaVendorAdapter();
    adapter.setShowSql(properties.isShowSql());
    adapter.setGenerateDdl(properties.isGenerateDdl());
    return adapter;
}

I keep all the default behaviour but just disable that auto-detecting dialect

And define the Dialect in application.properties:

spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.XXXXXX

But the point is that I am doubtful if Athena can be used with Hibernate as I cannot find there is an existing Dialect for Athena. So what I suggest are:

  1. Try the Dialect from another database which its SQL syntax is similar to Athena. It is not surprise that it will not work for some cases.

  2. Implement the Dialect for Athena by yourself.

like image 67
Ken Chan Avatar answered Sep 28 '22 11:09

Ken Chan