Menu
I have registertemptable in Apache Spark using Zeppelin below:
val hvacText = sc.textFile("...") case class Hvac(date: String, time: String, targettemp: Integer, actualtemp: Integer, buildingID: String) val hvac = hvacText.map(s => s.split(",")).filter(s => s(0) != "Date").map( s => Hvac(s(0), s(1), s(2).toInt, s(3).toInt, s(6))).toDF() hvac.registerTempTable("hvac") After I have done with my queries with this temp table, how do I remove it ?
I checked all docs and it seems I am getting nowhere.
Any guidance ?
329
DROP TABLE deletes the table and removes the directory associated with the table from the file system if the table is not EXTERNAL table. If the table is not present it throws an exception. In case of an external table, only the associated metadata information is removed from the metastore database.
In SQL Server, we can use the OBJECT_ID function to get the table name of the temporary table, and if the table is found, we can use the DROP TABLE statement to drop the temp table in sql.
registerTempTable (name)[source] Registers this DataFrame as a temporary table using the given name. The lifetime of this temporary table is tied to the SparkSession that was used to create this DataFrame . New in version 1.3. 0.
Spark 2.x
For temporary views you can use Catalog.dropTempView:
spark.catalog.dropTempView("df") For global views you can use Catalog.dropGlobalTempView:
spark.catalog.dropGlobalTempView("df") Both methods are safe to call if view doesn't exist and, since Spark 2.1, return boolean indicating if the operation succeed.
Spark 1.x
You can use SQLContext.dropTempTable:
scala.util.Try(sqlContext.dropTempTable("df")) It can be still used in Spark 2.0, but delegates processing to Catalog.dropTempView and is safe to use if table doesn't exist.
186
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
