Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Limit number of goroutine picking task in loop

I am running a loop - 24 times which calls a downstream that is not able to handle all the work supplied concurrently by the go-routines spawned, I want to limit that such that only a specific number (3 or 4) of go-routine gets executed. Sample code looks like below, if anyone can point me to the right pattern to fulfil this would be a great help

for i:=0; i<24; i++ {
   go callSomeDownStream()
}
like image 623
Arpit Avatar asked Jan 29 '26 17:01

Arpit


2 Answers

You can use the channel of empty structs to control the number of concurrent worker goroutines

const MAX_CONCURRENT_JOBS = 3

func main() {
    waitChan := make(chan struct{}, MAX_CONCURRENT_JOBS)

    for i:=0; i < 24; i++ {
        waitChan <- struct{}{}
        go func() {
            callSomeDownStream()
            <-waitChan
        }()
    }
}
like image 155
zangw Avatar answered Feb 01 '26 12:02

zangw


func callSomeDownStream(wg *sync.WaitGroup, queue chan struct{}) {
    defer func() {
        <-queue
        wg.Done()
    }()
    // do something
}

func main() {
    wg := sync.WaitGroup{}
    queue := make(chan struct{}, 3)
    for i := 0; i < 24; i++ {
        queue <- struct{}{}
        wg.Add(1)
        go callSomeDownStream(&wg, queue)
    }
    wg.Wait()
    close(queue)
}
like image 30
moyrne Avatar answered Feb 01 '26 13:02

moyrne



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!