Array size too big - ruby

Question

I am getting a 'ArgumentError: array size too big' message with the following code:

MAX_NUMBER = 600_000_000
my_array = Array.new(MAX_NUMBER)

Question. What is the max value that the Array.new function takes in Ruby?

Answer 1

An array with 500 million elements is 2 GiBytes in size, which – depending on the specific OS you are using – is typically the maximum that a process can address. In other words: your array is bigger than your address space.

So, the solutions are obvious: either make the array smaller (by, say, breaking it up in chunks) or make the address space bigger (in Linux, you can patch the kernel to get 3, 3.5 and even 4 GiByte of address space, and of course switching to a 64 bit OS and a 64 bit Ruby implementation(!) would also work).

Alternatively, you need to rethink your approach. Maybe use mmap instead of an array, or something like that. Maybe lazy-load only the parts you need.