Solr could not load MySQL JDBC Driver

Question

I am using Solr 4.6.1 on Mac. I try to figure out how to use data import to load data from MySQL. But I end up with fail to load JDBC Driver class. Here is what I did:

Put mysql-connector-java-5.1.29-bin.jar to ~/Tools/Solr/example/lib/

1.Configure ~/Tools/Solr/example/solr/collection1/conf/data-config.xml:

<dataConfig>
<dataSource type="JdbcDataSource" 
          driver="com.mysql.jdbc.Driver"
          url="jdbc:mysql://localhost:3306/test" 
          user="root" 
          password="root"/>
<document>
<entity name="SolrTest" 
        query="select * from SolrTest">
   <field column="ID" name="id"/>
   <field column="Name" name="name"/>
   <field column="Class" name="class"/>
   <field column="Score" name="score"/>
</entity>
</document>
</dataConfig>

2.Configure ~/Tools/Solr/example/solr/collection1/conf/solrconfig.xml:

add the following under < config> < /config>

<requestHandler name="/dataimport" 
                class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
    <str name="config">data-config.xml</str>
</lst>
</requestHandler>

also

<lib dir="../../../dist/" regex="solr-dataimporthandler-\d.*\.jar" />
<lib dir="../../../contrib/dataimporthandler/lib" regex=".*\.jar" />

3.Add the following to ~/Tools/Solr/example/solr/collection1/conf/schema.xml:

<field name="id" type="int" indexed="true" stored="true" required="true"/>
<field name="name" type="string" indexed="true" stored="true"/>
<field name="class" type="string" indexed="true" stored="true" />
<field name="score" type="int" indexed="true" stored="true"/>

Then run

java -jar start.jar

When I go to

http://localhost:8983/solr/#/collection1/dataimport//dataimport 

it shows

No information available (idle)

When I click the execute button, I got the following error message:

31537 [Thread-15] ERROR org.apache.solr.handler.dataimport.DataImporter – Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Could not load driver: com.mysql.jdbc.Driver Processing Document # 1 at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:476) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:457) Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Could not load driver: com.mysql.jdbc.Driver Processing Document # 1 at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:410) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:323) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:231) ... 3 more Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Could not load driver: com.mysql.jdbc.Driver Processing Document # 1 at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:71) at org.apache.solr.handler.dataimport.JdbcDataSource.createConnectionFactory(JdbcDataSource.java:116) at org.apache.solr.handler.dataimport.JdbcDataSource.init(JdbcDataSource.java:64) at org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance(DataImporter.java:383) at org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextImpl.java:99) at org.apache.solr.handler.dataimport.SqlEntityProcessor.init(SqlEntityProcessor.java:53) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:74) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:427) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:408) ... 5 more Caused by: java.lang.ClassNotFoundException: Unable to load com.mysql.jdbc.Driver or org.apache.solr.handler.dataimport.com.mysql.jdbc.Driver at org.apache.solr.handler.dataimport.DocBuilder.loadClass(DocBuilder.java:916) at org.apache.solr.handler.dataimport.JdbcDataSource.createConnectionFactory(JdbcDataSource.java:114) ... 12 more Caused by: org.apache.solr.common.SolrException: Error loading class 'com.mysql.jdbc.Driver' at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:470) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:401) at org.apache.solr.handler.dataimport.DocBuilder.loadClass(DocBuilder.java:906) ... 13 more Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:454) ... 15 more

How to solve this?

Answer 1

1. Create a folder inside your solr installation directory. (say solr-4.6.1/lib)

2. Place the mysql-connector-java-5.1.29-bin.jar inside the folder.

3. Edit solrconfig.xml and put the jar's path

   <lib dir="../../../lib/" regex="mysql-connector-java-5.1.29-bin.jar" />

4. Restart Solr.

Also please check your URL. It's having //

http://localhost:8983/solr/#/collection1/dataimport//dataimport 

Answer 2

So for those unfortunate souls who end up here. I'm using the latest Solr Docker image (8.4) which is configured so that the solr Unix-user doesn't have any permissions to create/delete folders and whatnot. For some reason I managed to create a lib file which I took for being a directory. After debugging a while I noticed this error, and promptly just changed the

<lib dir="${solr.install.dir:../../../..}/contrib/dataimporthandler/lib" regex=".*\.jar" />

to

<lib dir="${solr.install.dir:../../../..}/contrib/dataimporthandler/" regex=".*\.jar" />

where my postgresql-42.2.10.jar driver is. Otherwise all the other steps apply (edit solrconfig.xml, add data-config.xml, add schema.xml, copy posgresql-*.jar to the container's /opt/solr-8.4.1/contrib/dataimporthandler). Maybe one thing worth of mentioning is that I used Docker's internal network URL instead of localhost: url="jdbc:postgresql://host.docker.internal:5600/my_postgres_db" but I don't think it matters.

And oh, remember to restart Solr after adding the files / making changes =).