Creating Large Matrix in R [duplicate]

Question

How can I create a large matrix in R or how can I zip a large matrix into some object?

y <- matrix(rnorm(2e4*2e4),1:20000,1:20000)
Error: cannot allocate vector of size "..." Gb

How is it possible to allocate this matrix?

Answer 1

I'm not sure why I'm giving this a serious answer but here goes...

1.I can install bigmemory just fine. Perhaps if you are having problems you should share the error message, your environment and OS, and then ask for help on that instead.

2.Perhaps this is just an example:

  y <- outer(1:20000,1:20000)
  Error: cannot allocate vector of size "..." Gb

but do you really want to do this? Couldn't you refactor your code to iterate/resample a smaller matrix many times or read the data a bit at a time, to wit...

3.Have you considered using RSQlite? It's super easy to install and especially now with the dplyr package is very user friendly.

Answer 2

I am a huge fan of the bigmemory package and I am a Windows user. To use the package I first chose to stay with R 2.15 as it still has a functional Windows version of bigmemory package available on CRAN. For R 3.0 I had to compile that old version to work with my big matrices.