Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What values can be entered for `R_MAX_VSIZE` in R?

I am working on memory-intensive operations in R. I have seen some advice on this site about increasing the memory limit using R_MAX_VSIZE but I haven't found a clear explanation of exactly what sorts of values can be inputted and how they are read.

What formats are accepted/valid for the R_MAX_VSIZE environment variable?

Here is what I have found so far:

  1. R base documentation - Memory: Memory Available for Data Storage

    The maximal vector heap size can be set with the environment variable R_MAX_VSIZE

    I can't find additional information about what values are allowed on this help page.

  2. Graeme Frost's answer to R on MacOS Error: vector memory exhausted (limit reached?)

    This answer suggests setting the value to "100Gb". When I encountered the issue on my machine, this actually fixed my problem, but there are other answers that indicate other values are valid.

  3. Connor Dibble's answer to Error: vector memory exhausted (limit reached?) R 3.5.0 macOS

    This answer sets the value using export R_MAX_VSIZE=32000000000 which doesn't include the units, so I assume it is being interpreted as bytes.

like image 866
Joshua Shew Avatar asked Oct 13 '25 01:10

Joshua Shew


1 Answers

At least since R-devel at the time of writing this (4.5.0), the Memory chapter that covers R_MAX_VSIZE has been updated to include what its value should be:

Vector heap limits are given in bytes.

However, in R's source code (at least a mirror of it), we can see that the value retrieved from the environment variable is passed to R_Decode2Long:

if ((p = getenv("R_MAX_VSIZE"))) {
value = R_Decode2Long(p, &ierr);

In this function's definition, values with units (i.e. 100G, 20M, etc.) are also interpreted:

R_size_t R_Decode2Long(char *p, int *ierr)
{
    R_size_t v = strtol(p, &p, 10);
    *ierr = 0;
    if(p[0] == '\0') return v;
    /* else look for letter-code ending : */
    if(R_Verbose)
    REprintf("R_Decode2Long(): v=%ld\n", (long)v);
    // NOTE: currently, positive *ierr are not differentiated in the callers:
    if(p[0] == 'G') {
    if((Giga * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 4; return(v); }
    return (R_size_t) Giga * v;
    }
    else if(p[0] == 'M') {
    if((Mega * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 1; return(v); }
    return (R_size_t) Mega * v;
    }
    else if(p[0] == 'K') {
    if((1024 * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 2; return(v); }
    return (1024*v);
    }
    else if(p[0] == 'k') {
    if((1000 * (double)v) > (double) R_SIZE_T_MAX) { *ierr = 3; return(v); }
    return (1000*v);
    }
    else {
    *ierr = -1;
    return(v);
    }
}
like image 147
Justin Singh-M. Avatar answered Oct 14 '25 18:10

Justin Singh-M.