Igor Berman questions

Igor Berman answers

• Spark: cache RDD to be used in another job
• Spark Is there any rule of thumb about the optimal number of partition of a RDD and its number of elements?
• Cannot allocate 298930300 bytes of memory (of type "old_heap")
• Joining a large and a ginormous spark dataframe
• spark: java.io.IOException: No space left on device [again!]
• Replacing whitespace in all column names in spark Dataframe
• org.apache.avro.UnresolvedUnionException: Not in union ["long","null"]