R Error: java.lang.OutOfMemoryError: Java heap space

Question

I am trying to connect R to Teradata to pull data directly into R for analysis. However, I am getting the error of,

Error in .jcall(rp, "I", "fetch", stride, block) :   java.lang.OutOfMemoryError: Java heap space 

I have tried to set up my R options to increase the max heap size of JVM by doing:

options(java.parameters = "-Xmx8g") 

I have also tried to initialize java parameters with rJava function .jinit as: .jinit(parameters="-Xmx8g"). But still failed.

The calculated size of the data should be approximately 3G (actually less than 3G).

Answer 1

You need to make sure you're allocating additional memory before loading rJava or any other packages. Wipe the environment first (via rm(list = ls())), restart R/Rstudio if you must, and modify the options at the beginning of your script.

options(java.parameters = "-Xmx8000m")

See for example https://support.snowflake.net/s/article/solution-using-r-the-following-error-is-returned-javalangoutofmemoryerror-gc-overhead-limit-exceeded

Answer 2

I somehow had this problem in a not reproducible manner, partly solved it with -Xmx8g but run in to problems randomly.

I now found an option with a different garbage collector by using

options(java.parameters = c("-XX:+UseConcMarkSweepGC", "-Xmx8192m")) library(xlsx) 

at the beginning of the script and before any other package is loaded since other packages can load some java things by themselves and the options have to be set before any Java is loaded.

So far, the problem didn't occurred again.

Only sometimes in a long session it can still happen. But in this case a session restart normally solves the problem.