Remove Temporary Tables from Apache SQL Spark

Question

I have registertemptable in Apache Spark using Zeppelin below:

val hvacText = sc.textFile("...")  case class Hvac(date: String, time: String, targettemp: Integer, actualtemp: Integer, buildingID: String)  val hvac = hvacText.map(s => s.split(",")).filter(s => s(0) != "Date").map(     s => Hvac(s(0),              s(1),             s(2).toInt,             s(3).toInt,             s(6))).toDF()  hvac.registerTempTable("hvac") 

After I have done with my queries with this temp table, how do I remove it ?

I checked all docs and it seems I am getting nowhere.

Any guidance ?

Answer 1

Spark 2.x

For temporary views you can use Catalog.dropTempView:

spark.catalog.dropTempView("df") 

For global views you can use Catalog.dropGlobalTempView:

spark.catalog.dropGlobalTempView("df") 

Both methods are safe to call if view doesn't exist and, since Spark 2.1, return boolean indicating if the operation succeed.

Spark 1.x

You can use SQLContext.dropTempTable:

scala.util.Try(sqlContext.dropTempTable("df")) 

It can be still used in Spark 2.0, but delegates processing to Catalog.dropTempView and is safe to use if table doesn't exist.