Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using G1GC garbage collector with spark 2.3

Tags:

I am trying to use the G1GC garbage collector for spark job but I get a

Error: Invalid argument to --conf: -XX:+UseG1GC

I tried using these options but haven't been able to get it working

spark-submit --master spark://192.168.60.20:7077 --conf -XX:+UseG1GC /appdata/bblite-codebase/test.py

and

spark-submit --master spark://192.168.60.20:7077 -XX:+UseG1GC /appdata/bblite-codebase/test.py

What is the correct way to call a G1GC collector from spark?