Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why goroutine leaks

Tags:

go

goroutine

I read Twelve Go Best Practices and encounter and interesting example on page 30.

func sendMsg(msg, addr string) error {
    conn, err := net.Dial("tcp", addr)
    if err != nil {
        return err
    }
    defer conn.Close()
    _, err = fmt.Fprint(conn, msg)
    return err
} 

func broadcastMsg(msg string, addrs []string) error {
    errc := make(chan error)
    for _, addr := range addrs {
        go func(addr string) {
            errc <- sendMsg(msg, addr)
            fmt.Println("done")
        }(addr)
    }

    for _ = range addrs {
        if err := <-errc; err != nil {
            return err
        }
    }
    return nil
}

func main() {
    addr := []string{"localhost:8080", "http://google.com"}
    err := broadcastMsg("hi", addr)

    time.Sleep(time.Second)

    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println("everything went fine")
}

The programmer mentioned, that happens to the code above:

the goroutine is blocked on the chan write
the goroutine holds a reference to the chan
the chan will never be garbage collected

Why the goroutine is blocked here? The main thread is blocked, until it receive data from goroutine. After it continues the for loop. Not?

Why the errc chan will be never garbage collected? Because I do not close the channel, after goroutine is finished?

like image 795
softshipper Avatar asked Apr 27 '15 10:04

softshipper


People also ask

How do you stop a Goroutine leak?

We can avoid the problem by signaling the internal goroutine with a stop channel but there is a better solution: cancellable contexts. The generator can select on a context's Done channel and once the context is done, the internal goroutine can be cancelled.

What are Goroutine leaks?

When a new Goroutine is created, computers allocate memories in heap and release them once the execution is finished. A Goroutine leak is a memory leak that occurs when a Goroutine is not terminated and is left hanging in the background for the application's lifetime.

How do you know if a Goroutine is leaking?

Identifying Goroutine Leaks There are 2 ways to detect leaks. Using some sort of profiler to profile the code and analyse, you can check for non-terminating goroutines. A couple of examples of such profilers are net/http/pprof and github.com/google/gops.

What is the use of Goroutine?

A goroutine is a function that executes simultaneously with other goroutines in a program and are lightweight threads managed by Go. A goroutine takes about 2kB of stack space to initialize.


2 Answers

One problem I see is that inside broadcastMsg() after goroutines have started:

for _ = range addrs {
    if err := <-errc; err != nil {
        return err
    }
}

If a non-nil error is received from errc, broadcastMsg() returns immediately with that error and does not receive futher values from the channel, which means further goroutines will never get unblocked because errc is unbuffered.

Possible Fixes

A possible fix would be to use a buffered channel, big enough to not block any of the goroutines, in this case:

errc := make(chan error, len(addrs))

Or even if a non-nil error is received from the channel, still proceed to receive as many times as many goroutines send on it:

var errRec error
for _ = range addrs {
    if err := <-errc; err != nil {
        if errRec == nil {
            errRec = err
        }
    }
}
return errRec

Or as mentioned in the linked talk on slide #33: use a "quit" channel to prevent the started goroutines to remain blocked after broadcastMsg() has completed/returned.

like image 186
icza Avatar answered Sep 21 '22 07:09

icza


You have a list of two addresses (localhost, google). To each of these you're sending a message (hi), using one goroutine per address. And the goroutine sends error (which may be nil) to errc channel.

If you send something to a channel, you also need something that reads the values from that channel, otherwise it will block (unless it's a buffered channel, but even buffered channels block once their buffer is full).

So your reading loop looks like this:

for _ = range addrs {
    if err := <-errc; err != nil {
        return err
    }
}

If the first address returns an error which is not nil, the loop returns. The subsequent error values are never read from the channel thus it blocks.

like image 40
frostschutz Avatar answered Sep 21 '22 07:09

frostschutz