Boto3 athena query without saving data to s3

Question

I am trying to use boto3 to run a set of queries and don't want to save the data to s3. Instead I just want to get the results and want to work with those results. I am trying to do the following

import boto3

client = boto3.client('athena')
response = client.start_query_execution(
        QueryString='''SELECT * FROM mytable limit 10''',
        QueryExecutionContext={
            'Database': 'my_db'
            }.        
        ResultConfiguration={
            'OutputLocation': 's3://outputpath',
            }
        )

print(response)

But here I don't want to give ResultConfiguration because I don't want to write the results anywhere. But If I remove the ResultConfiguration parameter I get the following error

botocore.exceptions.ParamValidationError: Parameter validation failed:
Missing required parameter in input: "ResultConfiguration"

So it seems like giving s3 output location for writing is mendatory. So what could the way to avoid this and get the results only in response?

Answer 1

You will have to specify an S3 temp bucket location whenever running the 'start_query_execution' command. However, you can get a result set (a dict) by running the 'get_query_results' method using the query id.

The response (dict) will look like this:

{
'UpdateCount': 123,
'ResultSet': {
    'Rows': [
        {
            'Data': [
                {
                    'VarCharValue': 'string'
                },
            ]
        },
    ],
    'ResultSetMetadata': {
        'ColumnInfo': [
            {
                'CatalogName': 'string',
                'SchemaName': 'string',
                'TableName': 'string',
                'Name': 'string',
                'Label': 'string',
                'Type': 'string',
                'Precision': 123,
                'Scale': 123,
                'Nullable': 'NOT_NULL'|'NULLABLE'|'UNKNOWN',
                'CaseSensitive': True|False
            },
        ]
    }
},
'NextToken': 'string'
}

For more information, see boto3 client doc: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena.html#Athena.Client.get_query_results

You can then delete all files in the S3 temp bucket you've specified.