moriarty007 questions

moriarty007 answers

• having Spark process partitions concurrently, using a single dev/test machine
• Multiple SparkSessions in single JVM
• Git: Cherry-Pick to working copy without commit
• Efficient way to read specific columns from parquet file in spark
• Spark FileAlreadyExistsException on Stage Failure
• How to change number of executors in local mode?
• Does reducing the number of executor-cores consume less executor-memory?
• Create an empty array column of certain type in pyspark DataFrame
• Hive partitioned table reads all the partitions despite having a Spark filter