Spark executor GC taking long

Question

Am running Spark job on a standalone cluster and I noticed after sometime the GC starts taking long and the red scary color begins to show.

Here is the resources available:

Cores in use: 80 Total, 76 Used
Memory in use: 312.8 GB Total, 292.0 GB Used

Job details:

spark-submit   --class com.mavencode.spark.MonthlyReports   
--master spark://192.168.12.14:7077   
--deploy-mode cluster   --supervise   
--executor-memory 16G --executor-cores 4 
--num-executors 18  --driver-cores 8 
--driver-memory 20G montly-reports-assembly-1.0.jar

How do I fix the GC time taking so long?

Answer 1

I had the same problem and could resolve it by using Parallel GC instead of G1GC. You may add the following options to the executors additional Java options in the submit request

-XX:+UseParallelGC -XX:+UseParallelOldGC