Migrating Solr from 3.6 to 4.0, method to do it, and is it safe?

Question

We are using Solr 3.6 in master-slave configuration, 100s of cores, 100s of millions documents and need round-the-clock uptime. We wish to upgrade to Solr 4.0.

---

I believe the correct method is to do a filesystem copy of the old indexes to the new location and configuring SolrConfig with:

<luceneMatchVersion>LUCENE_33</luceneMatchVersion>

Question 1: Will this make Solr use the obsolete format? Will this convert the index to the 4.0 format in an online manner?

---

Question 2: Is there a better method that needs lesser downtime to convert the indexes? Can I replicate from the old server to the new?

---

Question 3: And can anyone confirm if Solr 4.0 is safe for as heavy use as ours?

Answer 1

If you keep

<luceneMatchVersion>LUCENE_33</luceneMatchVersion>

instead of using:

<luceneMatchVersion>LUCENE_40</luceneMatchVersion>

you will not get the full benefits of Solr 4 / Lucene 4. However you cannot switch to LUCENE_40 without rebuilding all your indexes. If you stick with LUCENE_33 it will still use the old index format.

---

For question 2, if you have enough resources, set up a second instance of Solr and copy indexes to Solr4. This can be done by writing a Java program with SolrJ. Not sure of any existing ones out there, but there probably are.

Advantage: you can do it live.

Disadvantage: doesn't work well if your Solr 3 instance is constantly being updated.

If you have a timestamp field that records the date/time each record was updated, similar to the following:

<field name="timestamp" type="date" indexed="true" stored="true" default="NOW" multiValued="false"/>

this can be easier as when you are reading you can do a query where the timestamp is less than the current date, whose results are less likely to change as you are iterating over results (though not impossible if existing records are updated or deleted).

---

We have been running Solr 4.0 for over four months now without issues. It's a single Solr core that has 150 million documents.