Storing arrays in memory and using these arrays later

Question

I am currently working on a program which requires preprocessing; filling multidimensional arrays with around 5765760*2 values.

My issue is that I have to run this preprocessing every time before I actually get to test the data and it takes around 2 minutes.

I don't want to have to wait 2 minutes each time I run a test, but I also don't want to store the values in a file.

Is there a way to store the values in a temporary memory rather than actually outputting them into a file?

Answer 1

I think, what you are asking for translates to: "can I make my JVM write data to some place in memory so that another JVM instance can later on read from there?"

And the simple answer is: no, that is not possible.

When the JVM dies, the memory consumed by the JVM is returned to the OS. That stuff is gone.

So even the infamous sun.misc.Unsafe with "direct" memory access does not allow you to do that.

The one thing that would work: if your OS is Linux, you could create a RAM disc. And then you write your file to that.

So, yes, you store your data in a file, but the file resides in memory; thus reading/writing is much faster compared to disk IO. And that data stays available as long as you don't delete the RAM disc or restart your OS.

On the other hand, when your OS is Linux, and you have enough RAM (a few GB should do!) then you should just try if an "ordinary disc" isn't good enough.

You see - those modern OSes, they do a lot of things in the background. It might look like "writing to disk", but in the end, the Linux OS just keeps using the memory.

So, before you spent hours on bizarre solutions - measure the impact of writing to disk!