EMR conf spark-default settings

Question

I am using configuration file according to guides Configure Spark to setup EMR configuration on AWS, for example, changing the spark.executor.extraClassPath is via the following settings:

{
     "Classification": "spark-defaults",
     "Properties": {
         "spark.executor.extraClassPath": "/home/hadoop/mongo-hadoop-spark.jar",
     }
}

It works prefect and do change spark.executor.extraClassPath on emr spark conf, but emr has some preset default paths in spark.executor.extraClassPath, so instead of overwriting the spark.executor.extraClassPath.I would like to know if there is a way to append the path and keep the default paths such as

{
     "Classification": "spark-defaults",
     "Properties": {
         "spark.executor.extraClassPath": "{$extraClassPath}:/home/hadoop/mongo-hadoop-spark.jar",
     }
}

Answer 1

You can specify it in your emr template as follows

Classification: spark-defaults
          ConfigurationProperties:
            spark.jars: Your jar location