Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Caching of the data of a big file in memory in java

Hi I am working on Spelling Corrector project of Natural Language processing and I am supposed read data from a file whose size is 6.2 MB 1 GB. While it is working fine, the problem that I am facing is that every time I run the java program I have to load the data in to the memory and it is taking same amount of time every time it is run.

Is there any way this data can cached in to the memory in java?Can any one suggest me some work around of it?

Basically what I want to know is that What is procedure of storing content of a large file in memory so that I dont have to read it again? lets say file is of GB.

like image 501
Dude Avatar asked Dec 16 '22 20:12

Dude


2 Answers

6.2 MB of data will probably be stored in the cache of your operating system as it is a relatively small amount of data and therefore shouldn't take much time at all to load. You should investigate whether it is the parsing of this data that is taking a long time and maybe cache the parsed data to a binary file for quick loading.

like image 76
Sean Dawson Avatar answered Dec 21 '22 22:12

Sean Dawson


6.2 MB isn't very big and unless this is taking a long time and you can't use a background thread to load the file I wouldn't worry about it.

You can use memory mapped files but these are not as simple to work with. Memory mapped files are useful if you have between 1 GB and 1 TB of data.

like image 31
Peter Lawrey Avatar answered Dec 21 '22 22:12

Peter Lawrey