Shutdown spark structured streaming gracefully

Question

There is a way to enable graceful shutdown of spark streaming by setting property spark.streaming.stopGracefullyOnShutdown to true and then kill the process with kill -SIGTERM command. However I don't see such option available for structured streaming (SQLContext.scala).

Is the shutdown process different in structured streaming? Or is it simply not implemented yet?

Answer 1

This feature is not implemented yet. But the write ahead logs of spark structured steaming claims to recover state and offsets without any issues.