Since YARN is responsible of managing the resources of Hadoop applications, like cores and processes, is a YARN application allowed to create new threads ?
I refer mainly to Spark applications running on YARN resource manager. I know that if you create a new thread no errors are thrown but, is it safe ?
I reply to myself. YARN controls resources like memory and cores, not threads. Cores are controlled by changing OS specific parameters when creating containers (process affinity). Then, the JVM is running using a specified number of cores and you can create as many threads as you want.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With