Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Monitor memory usage in R

Tags:

r

Is it possible to monitor the amount of memory that is in use or has been used by R to call a function? For example, I have an arbitrary function, e.g:

smallest.sv <- function(){   A <- matrix(rnorm(1e6), 1e3);   mysvd <- svd(A);   return(tail(mysvd$d, 1)); } 

Running the function simply returns a scalar, but a lot of memory was used to calculate the function. Now I need to do performance benchmarking. Processing time is easy:

system.time(x <- smallest.sv()) 

However I would also like to know how much memory was needed for this call, without modifying the function (it should work for arbitrary functions). Is there any way to do this?

Edit: to clarify a bit. I am mostly interested in the upper bound of memory that was in use during the call of the function, i.e. how much physical memory is required to be able to process the function call. In many cases this is significantly less than the total amount of allocated memory I think.

like image 597
Jeroen Ooms Avatar asked Oct 21 '11 23:10

Jeroen Ooms


People also ask

How do I see memory usage in R?

It is roughly the sum of the size of all of the R objects in the global environment, plus libraries and packages that have been loaded and internal R and RStudio overhead. Click the triangle drop-down and choose "Memory Usage Report" to see a breakdown of your memory usage in more detail.

Does R have memory management?

So the gist of the matter is that R has been improving performance and memory management for a very long time.

What does gc () do in R?

R uses an alternative approach: garbage collection (or GC for short). GC automatically releases memory when an object is no longer used. It does this by tracking how many names point to each object, and when there are no names pointing to an object, it deletes that object.


1 Answers

R provides memory profiling support, see Section 3.3 of the Writing R Extensions manual :

3.3 Profiling R code for memory use

Measuring memory use in R code is useful either when the code takes more memory than is conveniently available or when memory allocation and copying of objects is responsible for slow code. There are three ways to profile memory use over time in R code. All three require R to have been compiled with `--enable-memory-profiling', which is not the default, but is currently used for the Mac OS X and Windows binary distributions. All can be misleading, for different reasons.

In understanding the memory profiles it is useful to know a little more about R's memory allocation. Looking at the results of `gc()' shows a division of memory into `Vcells' used to store the contents of vectors and `Ncells' used to store everything else, including all the administrative overhead for vectors such as type and length information. In fact the vector contents are divided into two pools. Memory for small vectors (by default 128 bytes or less) is obtained in large chunks and then parcelled out by R; memory for larger vectors is obtained directly from the operating system.

and then provides three more sections.

like image 74
Dirk Eddelbuettel Avatar answered Sep 17 '22 14:09

Dirk Eddelbuettel