How to dynamically decide on number of goroutines working on a task

Question

I have written a dummy code to demonstrate the purpose.

There are 2 channels and 3 goroutines in the code.

1 goroutine is generating numbers based on if they are divisible by 100 with no remainder:

• If the number is divisible by 100, it pushes it to the first channel.

• Otherwise it pushes it to the second channel.

2 goroutines are the consumers of these channels:

• 1 goroutine is responsible for consuming the number 1...99 - 101...199 etc.

• Other goroutine is responsible for 100, 200, 300 etc.

Now obviously, one goroutine has 99x more work to do than the other goroutine. How is this handled in Go? If a goroutine works more than other, is this goroutine given more CPU time? Or should I handle this situation, for example creating 99 goroutines for the more resource-hungry channel? (for the sake of argument, the jobs are thought of as identical)

func main() {
    ch1 := make(chan int)
    ch2 := make(chan int)

    go generator(ch1, ch2)
    go handler(ch1)
    go handler2(ch2)

    time.Sleep(1*time.Second)
}

func generator(chan1, chan2 chan int){
    for i:=0 ; ; i++{
        if i%100 == 0{
            chan1 <- i
        }else{
            chan2 <- i
        }
    }
}

func handler(number chan int){
    for _ = range number{
        num := <- number
        fmt.Println("Number divided by 100 is 0. ", num)
    }
}

func handler2(number chan int){
    for _ = range number{
        num := <- number
        fmt.Println("Number divided by 100 is not 0. ", num)
    }
}

Answer 1

How much CPU resource a goroutine gets depends on a lot of things.

What we can say in general is that the goroutine that only handles the numbers dividable by 100 will most likely wait a lot more than the other. You shouldn't worry about this, waiting for an element on a channel does not require CPU resources, so if you have "enough" other goroutines that have jobs to do, they can utilize your CPU.

Your example is simple for obvious reasons, but in a real-life example it would be more profitable to abstact your tasks into general tasks (e.g. handling any number could be a task), create and use a general worker pool, and send all tasks for execution to the pool. That way no matter how many goroutines the pool has, if there is work to do and there is a free (waiting) goroutine, it will take on the task, utilizing your CPU resource as much as possible. The job processor (executioner) should have the knowledge what to do with a number being 100 or 101.

For an example how such a goroutine pool can be implemented, see Is this an idiomatic worker thread pool in Go?