I am working on memory-intensive operations in R. I have seen some advice on this site about increasing the memory limit using R_MAX_VSIZE
but I haven't found a clear explanation of exactly what sorts of values can be inputted and how they are read.
What formats are accepted/valid for the R_MAX_VSIZE
environment variable?
Here is what I have found so far:
R base documentation - Memory: Memory Available for Data Storage
The maximal vector heap size can be set with the environment variable
R_MAX_VSIZE
I can't find additional information about what values are allowed on this help page.
Graeme Frost's answer to R on MacOS Error: vector memory exhausted (limit reached?)
This answer suggests setting the value to "100Gb". When I encountered the issue on my machine, this actually fixed my problem, but there are other answers that indicate other values are valid.
Connor Dibble's answer to Error: vector memory exhausted (limit reached?) R 3.5.0 macOS
This answer sets the value using export R_MAX_VSIZE=32000000000
which doesn't include the units, so I assume it is being interpreted as bytes.
At least since R-devel at the time of writing this (4.5.0), the Memory chapter that covers R_MAX_VSIZE
has been updated to include what its value should be:
Vector heap limits are given in bytes.
However, in R's source code (at least a mirror of it), we can see that the value retrieved from the environment variable is passed to R_Decode2Long
:
if ((p = getenv("R_MAX_VSIZE"))) {
value = R_Decode2Long(p, &ierr);
In this function's definition, values with units (i.e. 100G, 20M, etc.) are also interpreted:
R_size_t R_Decode2Long(char *p, int *ierr)
{
R_size_t v = strtol(p, &p, 10);
*ierr = 0;
if(p[0] == '\0') return v;
/* else look for letter-code ending : */
if(R_Verbose)
REprintf("R_Decode2Long(): v=%ld\n", (long)v);
// NOTE: currently, positive *ierr are not differentiated in the callers:
if(p[0] == 'G') {
if((Giga * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 4; return(v); }
return (R_size_t) Giga * v;
}
else if(p[0] == 'M') {
if((Mega * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 1; return(v); }
return (R_size_t) Mega * v;
}
else if(p[0] == 'K') {
if((1024 * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 2; return(v); }
return (1024*v);
}
else if(p[0] == 'k') {
if((1000 * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 3; return(v); }
return (1000*v);
}
else {
*ierr = -1;
return(v);
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With