Load entire Solr Instance in memory

Question

I am trying to load entire Solr instance in memory, I have 60 million records with 10 fields, (6 fields are indexed which icnlude DateTime, c_Text and string). My total solr instance size on disk is 15 GB, I have set the JVM Memory to:

Initial Memory Pool: 2048  (2 GB)
Maximum Memory Pool: 20480 (20 GB)

I changed the cache configuration in solrconfig.xml as:

<documentCache class="solr.LRUCache"
               size="6000000"
               initialSize="2000000"
               autowarmCount="0"/>

But still for each search it reads from the disk (checked through PerfMon utility on windows). Also in Task Manager it shows tomcat taking 500 MB of RAM. I have seen this question: Solr loads entire index into memory but it doesn't have an accepted answer and I also tried increase the cache size according to that question but still it doesn't increase the memory usage for tomcat.

I am using Solr 4.4 with Tomcat 7.0.42

Answer 1

You can use RAMDirectory to build the whole index into RAM.

This can be done in your solrconfig.xml

<config>

    <!-- ... -->

    <directoryFactory name="DirectoryFactory" 
        class="org.apache.solr.core.RAMDirectoryFactory"/> 

    <!-- ... -->

</config>

However with everything in live, this comes with Pros and Cons. A good elaboration can be read in the SO question Need to know pros and cons of using RAMDirectory.

You should then consider to decrease the size of your cache.