What's the best practice to synchronise wait groups and channels? I want to handle messages and block on a loop, and it appears that delegating the closing of the channel to another go routine seems to be a weird solution?
func Crawl(url string, depth int, fetcher Fetcher) {
ch := make(chan string)
var waitGroup sync.WaitGroup
waitGroup.Add(1)
go crawlTask(&waitGroup, ch, url, depth, fetcher)
go func() {
waitGroup.Wait()
close(ch)
}()
for message := range ch {
// I want to handle the messages here
fmt.Println(message)
}
}
func crawlTask(waitGroup *sync.WaitGroup, ch chan string, url string, depth int, fetcher Fetcher) {
defer waitGroup.Done()
if depth <= 0 {
return
}
body, urls, err := fetcher.Fetch(url)
if err != nil {
ch <- err.Error()
return
}
ch <- fmt.Sprintf("found: %s %q\n", url, body)
for _, u := range urls {
waitGroup.Add(1)
go crawlTask(waitGroup, ch, u, depth-1, fetcher)
}
}
func main() {
Crawl("http://golang.org/", 4, fetcher)
}
// truncated from https://tour.golang.org/concurrency/10 webCrawler
In order to be able to run the goroutines until the finish, we can either make use of a channel that will act as a blocker or we can use waitGroups that Go's sync package provides us with.
Also, channels use sync underneath thus using sync should be more performant. WaitGroup helps when you have to block wait for many goroutines to return. It's simpler when you might spawn a hundred of them in a for loop.
sync. WaitGroup provides a goroutine synchronization mechanism in Golang , and is used for waiting for a collection of goroutines to finish.
As an alternative to using a waitgroup and extra goroutine, you can use a separate channel for ending goroutines.
This is (also) idiomatic in Go. It involves blocking using a select
control group.
So you'd have to make
a new channel, typically with an empty struct as it's value (eg closeChan := make(chan struct{})
which, when closed (close(closeChan)
) would end the goroutine itself.
Instead of ranging over a chan
, you can use a select to block until either fed data or closed.
The code in Crawl
could look something like this:
for { // instead of ranging over a to-be closed chan
select {
case message := <-ch:
// handle message
case <-closeChan:
break // exit goroutine, can use return instead
}
}
And then in crawlTask
, you could close the closeChan
(passed in as another parameter, like ch
when you return (I figure that's when you want the other goroutine to end, and stop handling messages?)
if depth <= 0 {
close(closeChan)
return
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With