Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What does "WITH SERDEPROPERTIES ( 'paths' = 'key1, key2, key3') " really do in Hive DDL json serde?

Much appreciated if anyone can provide a reference to this clause. I have been searching online with little luck.

like image 347
Da Qi Avatar asked Feb 10 '17 23:02

Da Qi


2 Answers

It does nothing. There is nothing in the code of the serde that reads the path property – and as far as I can tell there has never been. This is probably documentation that has been handed down generation by generation from the earliest Hive examples.

like image 131
Theo Avatar answered Sep 28 '22 02:09

Theo


I contacted Amazon with this:

There is some documentation that says: with serdeproperties ( 'paths'='requestBeginTime, adId, impressionId, referrer, userAgent, userCookie, ip' ) This stackoverflow: What does "WITH SERDEPROPERTIES ( 'paths' = 'key1, key2, key3') " really do in Hive DDL json serde? Seems to say that is not needed. Can you remove it or explain it?

and they replied:

Thanks for your feedback. I confirmed that the WITH SERDEPROPERTIES line is not required for the example documented. We’ll get the line removed. Thanks once again for your help in improving the Athena documentation.

like image 43
markthegrea Avatar answered Sep 28 '22 03:09

markthegrea