Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

save dataframe as external hive table

I have used one way to save dataframe as external table using parquet file format but is there some other way to save dataframes directly as external table in hive like we have saveAsTable for managed table

like image 712
codeogeek Avatar asked May 31 '16 10:05

codeogeek


People also ask

What is the function you need to use to save a Dataframe as a Hive table?

DataFrames can also be saved as persistent tables into Hive metastore using the saveAsTable command.

How do you know if a Hive table is internal or external?

We can identify the internal or External tables using the DESCRIBE FORMATTED table_name statement in the Hive, which will display either MANAGED_TABLE or EXTERNAL_TABLE depending on the table type.


2 Answers

you can do this in this way

df.write.format("ORC").options(Map("path"-> "yourpath")) saveAsTable "anubhav"

like image 50
anubhav Avatar answered Nov 15 '22 07:11

anubhav


In PySpark, External Table can be created as below:

df.write.option('path','<External Table Path>').saveAsTable('<Table Name>')
like image 43
Ankur Avatar answered Nov 15 '22 08:11

Ankur