Can't use "update" in outputMode() when writing stream data in spark

Question

I'm trying to write stream data in spark to delta format, but it looks like it won't allow me to use update in outputMode(), below is my code and error message:

deltaStreamingQuery = (eventsDF
  .writeStream
  .format("delta")
  .option("checkpointLocation", checkpointPath)
  .outputMode("update")
  .queryName("stream_1p")
  .start(writePath)
)

AnalysisException: 'Data source com.databricks.sql.transaction.tahoe.sources.DeltaDataSource does not support Update output mode;'```

Answer 1

Currently Databricks Delta only supports append and complete as outputMode for sinks. append will add new rows to the table and complete will overwrite the table so perhaps this is what you are looking for to incorporate updates.

The official documentation is here => https://docs.databricks.com/delta/delta-streaming.html