How does Ray handles a number of jobs higher than the number of resources?

Question

Pretty basic question, but I wasn't able to find the answer in the docs. I am developing a computationally intensive application in Python and I'm employing Ray to parallelize computation. I only use remote functions (thus no Actors) and I have 40 cores available.

What happens when the main script sends a number of tasks higher than 40? Is Ray able to handle it or should I always control the number of tasks in order to keep it under the number of available cores?

Answer 1

In this scenario, Ray will queue the tasks, and run them as CPUs become available.

For example,

import ray 
import time

ray.init(num_cpus=10)

@ray.remote
def outer_task():
 return ray.get([inner_task.remote() for _ in range(20)])

@ray.remote
def inner_task():
 return time.sleep(1)

ray.get([outer_task.remote() for _ in range(20)])

in this scenario, there are 420 tasks which each require a CPU. Ray will queue and run these tasks, so that at most 10 tasks are running at the same time, and will make sure they all finish (in roughly 40 seconds).