Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Athena query fails with boto3 (S3 location invalid)

I'm trying to execute a query in Athena, but it fails.

Code:

client.start_query_execution(QueryString="CREATE DATABASE IF NOT EXISTS db;",
                           QueryExecutionContext={'Database': 'db'},
                           ResultConfiguration={
                                     'OutputLocation': "s3://my-bucket/",
                                     'EncryptionConfiguration': {
                                             'EncryptionOption': 'SSE-S3'
                                             }
                                     })

But it raises the following exception:

botocore.errorfactory.InvalidRequestException: An error occurred (InvalidRequestException) 
when calling the StartQueryExecution operation: The S3 location provided to save your 
query results is invalid. Please check your S3 location is correct and is in the same 
region and try again. If you continue to see the issue, contact customer support 
for further assistance.

However, if I go to the Athena Console, go to Settings and enter the same S3 location (for example):

athena settings

the query runs fine.

What's wrong with my code? I've used the API of several the other services (eg, S3) successfully, but in this one I believe I'm passing some incorrect parameters. Thanks.

Python: 3.6.1. Boto3: 1.4.4

like image 784
Diego Jancic Avatar asked Jul 25 '17 21:07

Diego Jancic


People also ask

How do I query S3 with Athena?

Go to the S3 bucket where source data is stored and click on the file. Click on the Copy Path button to copy the S3 URI for file. Navigate to AWS Athena service. In the Settings tab on top-right, enter S3 Bucket name where results of Athena queries will be stored, then click Save.

Why do I get the access denied error when I run a query in Amazon Athena?

You can get an "Access Denied" error because of the following reasons: The AWS Identity and Access Management (IAM) user doesn't have one or more of the following permissions: Read the source data bucket. Write the results to the query result bucket.

Does Athena work with S3?

Athena works directly with data stored in S3. Athena uses Presto, a distributed SQL engine to run queries. It also uses Apache Hive to create, drop, and alter tables and partitions.

Where is query result location in Athena?

If you run the query using the Athena console, the Query result location entered under Settings in the navigation bar determines the client-side setting. If you run the query using the Athena API, the OutputLocation parameter of the StartQueryExecution action determines the client-side setting.


1 Answers

I had to add a 'athena-' prefix to my bucket to get it to work. For example, in stead of:

"s3://my-bucket/"

Try:

"s3://athena-my-bucket/"

like image 53
Justin Domnitz Avatar answered Nov 06 '22 16:11

Justin Domnitz