DaRkMaN questions

DaRkMaN answers

• Any update about HIVE_STATS_JDBC_TIMEOUT and how to skip it in source level
• Cannot send Python dependencies to Spark on EMR via Livy
• what does load() do in spark?
• How to change query plan before execution (possibly turning an optimization off)?
• What triggers Jobs in Spark?
• Spark Scala: mapPartitions in this use case
• having Spark process partitions concurrently, using a single dev/test machine
• Disable spark catalyst optimizer
• Apache Spark 2.2: broadcast join not working when you already cache the dataframe which you want to broadcast
• Getting the number of rows in a Spark dataframe without counting