How to choose the queue for Spark job using spark-submit?

Question

Is there a way to provide parameters or settings to choose the queue in which I'd like my spark_submit job to run?

Answer 1

By using --queue So an example of a spark-submit job would be:-

spark-submit --master yarn --conf spark.executor.memory=48G --conf spark.driver.memory=6G --packages [packages separated by ,] --queue [queue_name] --class [class_name] [jar_file] [arguments]