Michael Heil questions

Michael Heil answers

• How to use transactions in Kafka and how to use abortTransaction?
• When does Kafka rotates partition leader? Do we have any property to control it?
• Unable to overwrite default value of "spark.sql.shuffle.partitions" with Spark Structured Streaming
• Kafka Producer guarantees
• Asynchronous auto-commit of offsets fails
• How to write streaming DataFrame into multiple sinks in Spark Structured Streaming
• How to use widgets to pass dynamic column names in Dataframe select statement
• Read latest message from kafka - segmentio/kafka-go
• When is it appropriate to use a UDF vs using spark functionality? [closed]
• KafkaConsumer.commitAsync() behavior with a lower offset than previous