Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

converting a struct to a json when querying athena

I have an athena table which I did not create or manage, but can query. one of the fields is a struct type. for the sake of the example let's suppose it looks like this:

my_field struct<a:string,
                b:string,
                c:struct<d:string,e:string>
                >

Now, I know how to query specific fields within this struct. But in one of my queries I need to extract the complete struct. so I just use:

select my_field from my_table

and the result looks like a string:

{a=aaa, b=bbb, c={d=ddd, e=eee}}

I want to get the result as a json string:

{"a":"aaa", "b":"bbb","c":{"d":"ddd", "e":"eee"}}

this string will then be processed by another application, this is why i need it in json format.

How can I achieve this?

EDIT: Better still, is there a way to query the struct in a way that flattens it? so the result would look like:

a   |   b   |   c.d  |  c.e   |
-------------------------------
aaa |   bbb |   ddd  |  eee   |
like image 903
amit Avatar asked Mar 03 '18 07:03

amit


People also ask

Can Athena query JSON?

Amazon Athena lets you parse JSON-encoded values, extract data from JSON, search for values, and find length and size of JSON arrays.

How do I query JSON?

To query JSON data, you can use standard T-SQL. If you must create a query or report on JSON data, you can easily convert JSON data to rows and columns by calling the OPENJSON rowset function. For more information, see Convert JSON Data to Rows and Columns with OPENJSON (SQL Server).

What is struct data type in Athena?

When you use CREATE_TABLE , Athena defines a STRUCT in it, populates it with data, and creates the ROW data type for you, for each row in the dataset. The underlying ROW data type consists of named fields of any supported SQL data types.


1 Answers

You can directly reference nested fields with a parent_field.child_field notation. Try:

SELECT
  my_field,
  my_field.a,
  my_field.b,
  my_field.c.d,
  my_field.c.e
FROM 
  my_table
like image 111
James Avatar answered Oct 04 '22 14:10

James