How to configure Yarn to use all vcores?

Question

We are running a spark streaming job using yarn as cluster manager, i have dedicated 7 cores per node to each node ...via yarn-site.xml as shown in the pic below

when the job is running ..it's only using 2 vcores and 5 vcores are left alone and the job is slow with lot of batches queued up ..

how can we make it use all the 7 vcores ..that's available to it this is usage when running so that it speed's up our job

Would greatly appreciate if any of the experts in the community will help out as we are new to Yarn & Spark

Answer 1

I searched many answers for this question. Finally, it worked after changing a yarn config file: capacity-scheduler.xml

<property>
 <name>yarn.scheduler.capacity.resource-calculator</name>
 <value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
</property>

Don't forget to restart your yarn