Apache solr configuration with tomcat 6.0

Question

Can you help me configuring Apache Solr using Tomcat and how to index in MS SQL database using Solr. What are the steps to configure Tomcat to run Apache Solr in Tomcat.

Answer 1

Here is the step by step procedure that would help.

PART 1: SETTING UP SOLR with TOMCAT

Step 1: Download Solr. It's just a zip file.

Step 2: Copy from your SOLR_HOME_DIR/dist/apache-solr-1.3.0.war to your tomcat webapps directory: $CATALINA_HOME/webapps/solr.war – Note the war file name change. That’s important.

Step 3: Create your solr home directory at a location of your choosing. This is where the configuration for that solr install resides. The easiest way to do this is to copy the SOLR_HOME_DIR/examples/solr directory to wherever it is you want your solr home container to be. Say place it in C:\solr.

Step 4: Hope you have set your environment variables, if not then please set JAVA_HOME, JRE_HOME, CATALINA_OPTS, CATALINA_HOME. Note that CATALINA_HOME refers to your Tomcat directory & CATALINA_OPTS refers to the amount of heap memory you want to give to your Solr.

Step 5: Start tomcat. Note this is only necessary to allow tomcat to unpack your war file. If you look under $CATALINA_HOME/webapps there should now be a solr directory.

Step 6: Stop tomcat

Step 7: Go into that solr directory and edit WEB-INF/web.xml. Scroll down until you see an entry that looks like this:

<!-- People who want to hardcode their "Solr Home" directly into the
     WAR File can set the JNDI property here...
 -->
<!--
  <env-entry>
     <env-entry-name>solr/home</env-entry-name>
     <env-entry-value>/Path/To/My/solr/Home/solr/</env-entry-value>
     <env-entry-type>java.lang.String</env-entry-type>
  </env-entry>
-->

Set your Solr home (for example: C:\solr) and uncomment the env entry.

Step 8: Start Tomcat again, and things should be going splendidly. You should be able to verify that solr is running by trying the url http://localhost:8080/solr/admin/.

PART 2: SETTING UP SOLR WITH MSSQL SERVER USING DATA IMPORT HANDLER

Step 1: Download Microsoft SQL Server JDBC Driver 3.0. Just extract the contents. Create a folder under your solr home directory (Example: C:\solr\lib). Copy the file sqljdbc4.jar out of the archive downloaded above into it.

Step 2: So under your Solr home the basic directories needed are conf and lib. The first one i.e. conf you might have got with Step 3 of part 1 & lib is the directory you have created in Step 1 of part 2.

Step 3. Go to conf directory. Please open 3 files in your editor: data-config.xml, schema.xml & solrconfig.xml.

Step 4. Start by editing data-config.xml. Place your SQL query, DB Name, Server name etc. For an example:

•   <dataConfig>
•   <dataSource type="JdbcDataSource" driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" url="jdbc:sqlserver://X.Y.Z.U:1433;databaseName=myDB" user="test" password="tester" /> 
•   <document>
•   <entity name="Text" query="select DocumentId, Data from Text">
•   <field column="DocumentId" name="DocumentId" /> 
•   <field column="Data" name="Data" /> 
•   </entity>
•   </document>
•   </dataConfig>

Step 5: Tell Solr about our data-config.xml file. This would be done by adding a request handler to the solrconfig.xml file which is solr configuration file. Add the following requesthandler to solrconfig.xml:

•   <requestHandler name="/dataimport"   class="org.apache.solr.handler.dataimport.DataImportHandler">
•   <lst name="defaults">
•   <str name="config">C:\solr\conf\data-config.xml</str> 
•   </lst>
•   </requestHandler>

Step 6: Configure schema.xml - In this file you can do several stuff like setting up datatypes of your fields, setting unique/primary key of your search etc.

Step 7: Start Tomcat

Step 8: Now visit http://localhost:8080/solr/admin/dataimport.jsp?handler=/dataimport & start your full import.

Some handy Notes:

    •   There are a number of reasons a data import could fail, most likely due to problem with
 the configuration of data-config.xml. To see for sure what's going on you'll have to look in
 C:\tomcat6\logs\catalina.*.

    •   If you happen to find that your import is failing due to system running out of memory,
 however, there's an easy, SQL Server specific fix. Add responseBuffering=adaptive and
 selectMethod=cursor to the url attribute of the dataSource node in data-config.xml. That stops the
 JDBC driver from trying to load the entire result set into memory before reads can occur.

    •   Note that by default the index gets created in C:\Tomcat6\bin\solr\data\index. To change this path
 just edit solrconfig.xml & change <dataDir>${solr.data.dir:./solr/data}</dataDir>.

    •   In new Solr versions, I think 3.0 and above you have to place the 2 data import handler
 jars in your solr lib directory (i.e. for example apache-solr-dataimporthandler-3.3.0.jar & apache-
solr-dataimporthandler-extras-3.3.0.jar). Search for them in your Solr zip you downloaded. In older
 Solr versions this is not required because they are bundled with solr.war. Since we have placed the
 data import handlers in the lib directory so we need to specify their paths in solrconfig.xml. Add
 this line to solrconfig.xml: (Example: <lib dir="C:/solr/lib/" regex="apache-solr-dataimporthandler-
\d.*\.jar" />)