Douglas M questions

Douglas M answers

• Is there a way to recover deleted data in Azure Databricks?
• use applyInPandas with PySpark on a cluster
• Is it possible to style an Entity Relation Diagram with Mermaid.js like Flowcharts to indicate a difference in entities?
• Databricks change default catalog
• Last Access Time Update in Hive metastore
• PySpark unzip files: Which is a good approach for unzipping files and storing the csv files into a Delta Table?
• Spark schema management at single place
• What is the optimal way to read from multiple Kafka topics and write to different sinks using Spark Structured Streaming?
• Deserialization of large numpy arrays using pickle is order of magnitude slower than using numpy
• Spark dataframe to arrow