Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does zip.NewReader(bytes.NewReader(), len) need to be closed or garbage collected automatically?

I'm having an argument with a friend, over possible wasted resources in Go.

Does a Reader have to be closed, if it's operating over an in-memory byte array?

func readJar(zipBytes []byte, readMeta bool) (m jar.Manifest, err error) {
    reader, err := zip.NewReader(bytes.NewReader(zipBytes), int64(len(zipBytes)))
    if err != nil {
        return
    }

    for _, file := range reader.File {
        switch file.Name {
        case jar.ManifestPath:
            m, err = readManifest(file)
            if err != nil {
                return
            }
        }
    }
    return
}

func readManifest(file *zip.File) (jar.Manifest, error) {
    reader, err := file.Open()
    if err != nil {
        return nil, err
    }

    defer reader.Close()
    return jar.ReadManifest(reader)
}

Whilst originally it was thought to be a source of File Handle Leaks, something else got the blame.

Will this leak memory, or does Go have sufficient escape analysis / garbage collection that it will be fine?

like image 864
Ryan Leach Avatar asked Jul 10 '18 09:07

Ryan Leach


People also ask

Does Golang have automatic garbage collection?

The Go standard toolchain provides a runtime library that ships with every application, and this runtime library includes a garbage collector.

Why does Go have garbage collection?

Garbage collection is a mechanism Go developers use to find memory space that is allocated recently but is no longer needed, hence the need to deallocate them to create a clean slate so that further allocation can be done on the same space or so that memory can be reused.

What is bytes buffer in Golang?

In go language, the buffer belongs to the byte package of the Go language, and we can use these package to manipulate the byte of the string. For example, suppose we have a string. We can read the length of the string with the len function, which will return the numeric length, but what if the strings are too large.


1 Answers

Golang compiler takes care for unreachable variables:-

The storage location does have an effect on writing efficient programs. When possible, the Go compilers will allocate variables that are local to a function in that function's stack frame. However, if the compiler cannot prove that the variable is not referenced after the function returns, then the compiler must allocate the variable on the garbage-collected heap to avoid dangling pointer errors. Also, if a local variable is very large, it might make more sense to store it on the heap rather than the stack.

Although Golang contains garbage collection. It would be better to use clean up functions. You can use defer function to close the file at the end of function.

defer f.close()

Check Documentation for SetFinalizer to get more understanding on these concepts:

func SetFinalizer(obj interface{}, finalizer interface{})

SetFinalizer sets the finalizer associated with obj to the provided finalizer function. When the garbage collector finds an unreachable block with an associated finalizer, it clears the association and runs finalizer(obj) in a separate goroutine. This makes obj reachable again, but now without an associated finalizer. Assuming that SetFinalizer is not called again, the next time the garbage collector sees that obj is unreachable, it will free obj.

The finalizer runs for an object to check if it is unreachable from the source. It can be used for file descriptors but it would be a mistake to depend on a finalizer to flush an in-memory I/O buffer such as a bufio.Writer, because the buffer would not be flushed at program exit.

like image 96
Himanshu Avatar answered Oct 03 '22 05:10

Himanshu