Menu
R has some tools for memory profiling, like Rprofmem(), Rprof() with option "memory.profiling=TRUE" and tracemem(). The last one can only be used on objects, and hence is useful to follow how many times an object is copied, but doesn't give an overview on a function basis. Rprofmem should be able to do that, but the output of even the simplest function call like lm() gives over 500 lines of log. I tried to figure out what Rprof("somefile.log",memory.profile=T) actually does, but I don't think I really get it.
The last I could find was this message of Thomas Lumley, saying that, and I quote :
I do not yet have tools to summarize the output.
This was in 2006. Any chance there are options for some nice summaries now, based on either Rprofmem(), the mysterious output of Rprof() with memory.profile set TRUE or any other tool?
374
Memory profiling enables you to understand the memory allocation and garbage collection behavior of your applications over time. It helps you identify method calls in the context within which most memory was allocated and combine this information with the number of allocated objects.
Because R stores all of its objects physical memory, it is important to be cognizant of how much memory is being used up by all of the data objects residing in your workspace. One situation where it's particularly important to understand memory requirements is when you are reading in a new dataset into R.
This is a python module for monitoring memory consumption of a process as well as line-by-line analysis of memory consumption for python programs. It is a pure python module which depends on the psutil module.
How Memory Profiling for C and C++ Works. When an application node is executed, the source code is instrumented by the C or C++ Instrumentor (attolcpp or attolcc1). The resulting source code is then executed and the Memory Profiling feature outputs a static . tsf file for each instrumented source file and a dynamic .
profvis looks like the the solution to this question.
It generates an interactive .html file (using htmlwidgets) showing the profiling of your code.
The introduction vignette is a good guide on its capability.
Taking directly from the introduction, you would use it like this:
devtools::install_github("rstudio/profvis") library(profvis) # Generate data times <- 4e5 cols <- 150 data <- as.data.frame(x = matrix(rnorm(times * cols, mean = 5), ncol = cols)) data <- cbind(id = paste0("g", seq_len(times)), data) profvis({ data1 <- data # Store in another variable for this run # Get column means means <- apply(data1[, names(data1) != "id"], 2, mean) # Subtract mean from each column for (i in seq_along(means)) { data1[, names(data1) != "id"][, i] <- data1[, names(data1) != "id"][, i] - means[i] } }, height = "400px") Which gives
183
Check out profr -- it seems like exactly what you're looking for.
33
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
