Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory leak with closed connections

I have an R application interacting with a Java daemon via stdin and stdout in an infinite loop, which seems to have some memory leaks. Simplified R app:

while (TRUE) {
    con <- file('stdin', open = 'r', blocking = TRUE)
    line <- scan(con, what = character(0), nlines = 1, quiet = TRUE)
    close(con)
}

This loop ends up using more and more RAM, and even if I manually gc() after the close(con) call, the memory footprint seems to be OK for a while, but eventually grows forever.

A basic script to confirm this:

Rscript --vanilla -e "while(TRUE)cat(runif(1),'\n')" | Rscript --vanilla -e "cat(Sys.getpid(), '\n');while (TRUE) {con <- file('stdin', open = 'r', blocking = TRUE);line <- scan(con, what = character(0), nlines = 1, quiet = TRUE);close(con);gc()}"

This will start two R processes: one writing to stdout and the other reading from stdin connected with a pipe (and the second printing the pid so that you can monitor the related memory usage):

enter image description here

I'm not sure what I'm doing wrong, but would love to stop this memory leak so any help is highly appreciated.

like image 997
daroczig Avatar asked Nov 10 '16 08:11

daroczig


People also ask

Does Closure cause memory leak?

In simple terms, a closure is an inner function that has access to the outer function's scope. In the example above, largeArray is never returned and cannot be reached by garbage collector, significantly increasing its size through repeated calls of inner functions, resulting in a memory leak.

How do you prevent a memory leak in closure?

Only capture variables as unowned when you can be sure they will be in memory whenever the closure is run, not just because you don't want to work with an optional self . This will help you prevent memory leaks in Swift closures, leading to better app performance.

What is the main cause of memory leaks?

A memory leak starts when a program requests a chunk of memory from the operating system for itself and its data. As a program operates, it sometimes needs more memory and makes an additional request.


1 Answers

Indeed (about reading about this on R-devel); notably, the memory leak has now been plugged in the development version of R, thanks to a patch by Gabor Csardi.

like image 111
Martin Mächler Avatar answered Oct 16 '22 10:10

Martin Mächler