Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Any better way to keep track of goroutine responses?

Tags:

go

goroutine

I'm trying to get my head around goroutines. I've created a simple program that performs the same search in parallel across multiple search engines. At the moment to keep track of the number of responses, I count the number I've received. It seems a bit amateur though.

Is there a better way of knowing when I've received a response from all of the goroutines in the following code?

package main

import (
    "fmt"
    "net/http"
    "log"
)

type Query struct {
    url string
    status string
}

func search (url string, out chan Query) {
    fmt.Printf("Fetching URL %s\n", url)
    resp, err := http.Get(url)

    if err != nil {
        log.Fatal(err)
    }

    defer resp.Body.Close()

    out <- Query{url, resp.Status}
}

func main() {
    searchTerm := "carrot"

    fmt.Println("Hello world! Searching for ", searchTerm)

    searchEngines := []string{
        "http://www.bing.co.uk/?q=",
        "http://www.google.co.uk/?q=",
        "http://www.yahoo.co.uk/?q="}

    out := make(chan Query)

    for i := 0; i < len(searchEngines); i++ {
        go search(searchEngines[i] + searchTerm, out)
    }

    progress := 0

    for {
                    // is there a better way of doing this step?
        if progress >= len(searchEngines) {
            break
        }
        fmt.Println("Polling...")
        query := <-out
        fmt.Printf("Status from %s was %s\n", query.url, query.status)
        progress++
    }
}
like image 257
Gly Jones Avatar asked Feb 23 '13 09:02

Gly Jones


People also ask

How do you manage Goroutines?

At the core of how I prefer to manage goroutines is a simple principle: When a routine starts another routine, then it is responsible for it. If nothing else, this means that a routine should always wait for all the others it has created. In Go this is mostly done using sync.

How do you know when a Goroutine is going to stop?

One goroutine can't forcibly stop another. To make a goroutine stoppable, let it listen for a stop signal on a dedicated quit channel, and check this channel at suitable points in your goroutine.

Is Goroutine a concurrency?

Goroutines are functions or methods that run concurrently with other functions or methods. Goroutines can be thought of as lightweight threads. The cost of creating a Goroutine is tiny when compared to a thread. Hence it's common for Go applications to have thousands of Goroutines running concurrently.

What happens when a Goroutine panics?

A panic stops the normal execution of a goroutine: When a program panics, it immediately starts to unwind the call stack. This continues until the program crashes and prints a stack trace, or until the built-in recover function is called.


1 Answers

Please use sync.WaitGroup, there is an example in the pkg doc

searchEngines := []string{
    "http://www.bing.co.uk/?q=",
    "http://www.google.co.uk/?q=",
    "http://www.yahoo.co.uk/?q="}
var wg sync.WaitGroup
out := make(chan Query)

for i := 0; i < len(searchEngines); i++ {
    wg.Add(1)
    go func (url string) {
        defer wg.Done()
        fmt.Printf("Fetching URL %s\n", url)
        resp, err := http.Get(url)

        if err != nil {
            log.Fatal(err)
        }

        defer resp.Body.Close()

        out <- Query{url, resp.Status}

    }(searchEngines[i] + searchTerm)

}
wg.Wait()
like image 70
Joe Avatar answered Sep 21 '22 19:09

Joe