Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to delete unmanaged delta lake table

I am trying to delete a delta lake table that was created using writestream. i tried drop table but it fails

#table created as
df.writestream().outputmode("append").format("delta").start("/mnt/mytable")

#attempt to drop table
spark.sql("drop table '/mnt/mytable'")
like image 266
user9297554 Avatar asked Oct 22 '25 14:10

user9297554


1 Answers

DROP TABLE IF EXISTS <unmanaged-table>    // deletes the metadata
dbutils.fs.rm("<your-s3-path>", true)   // deletes the data

DROP TABLE <managed-table> // deletes the metadata and the data

You need to specify the data to delete the data in an unmanaged table to because with an unmanaged table; Spark SQL only manages the meta data and you control the data location. With managed tables, Spark SQL manages both the metadata and the data and the data is stored in Databricks file system (DBFS) in your account. Thus, to delete an unmanaged table's data, you need to specify the path to the data.

like image 159
vivian Avatar answered Oct 25 '25 20:10

vivian



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!