Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

How to get Solr and CKAN to run on Ubuntu 18.04 after recent solr-jetty updates?

Tags:

solr

jetty

ckan

jetty-9

Problem

After doing some updates Jetty can no longer find Solr when trying to run CKAN on Ubuntu 18.04. A fresh install of CKAN is not working in development or prod.

This is using the 3.6.2+dfsg-18~18.04 package.

I can tell Jetty9 is running but it can't find solr.

Any help or pointers would be great.

Error Message

WARNI [pysolr] Unable to extract error message from invalid XML: mismatched tag: line 10, column 2
ERROR [pysolr] Solr responded with an error (HTTP 404): [Reason: None]
<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"/><title>Error 404 Not Found</title></head><body><h2>HTTP ERROR 404</h2><p>Problem accessing /solr/select/. Reason:<pre>    Not Found</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.15.v20190215</a><hr/></body></html>
ERROR [ckan.controllers.package] Dataset search error: ('SOLR returned an error running query: {\'sort\': \'score desc, metadata_modified desc\', \'fq\': [u\'\', u\'+site_id:"default"\', \'+state:active\', u\'+permission_labels:("public" OR "creator-aedc3c62-8492-48b7-9640-0c362bb2b537")\'], \'facet.mincount\': 1, \'rows\': 21, \'facet.field\': [u\'organization\', u\'groups\', u\'tags\', u\'res_format\', u\'license_id\'], \'facet.limit\': \'50\', \'facet\': \'true\', \'q\': \'*:*\', \'start\': 0, \'wt\': \'json\', \'fl\': \'id validated_data_dict\'} Error: SolrError(u\'Solr responded with an error (HTTP 404): [Reason: None]\\n<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"/><title>Error 404 Not Found</title></head><body><h2>HTTP ERROR 404</h2><p>Problem accessing /solr/select/. Reason:<pre>    Not Found</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.15.v20190215</a><hr/></body></html>\',)',)
INFO  [ckan.lib.base]  /dataset render time 0.079 seconds

Steps to Reproduce

  1. Create a new Ubuntu Server 18.04 LTS setup
  2. Follow the default source install documentation
  3. Start the server paster serve /etc/ckan/default/development.ini
  4. Navigate to the /dataset page

Background

This was working in February without issues by following the docs (which includes the comment from here).

I have noticed that in the newest solr-jetty package it adds the symlink by default which will mean the step to add this in the CKAN docs will fail as the symlink already exists. I've tried removing and adding it back to get the different permissions on the file with no luck.

Also, the first step under Solr setup instructs to update lines in the /etc/default/jetty file that no longer exist. I believe these should be moved to /etc/jetty9/start.ini but i've done both with no luck either.

UPDATES

  • Tried downgrading the solr-jetty package but same result (however symlink step worked as that wasn't included in the previous package).
    • sudo apt install libsolr-java=3.6.2+dfsg-11 solr-common=3.6.2+dfsg-11 solr-jetty=3.6.2+dfsg-11
    • sudo apt-get install python-dev postgresql libpq-dev python-pip python-virtualenv git-core redis-server
  • Tried downgrading both solr-jetty and openjdk-8-jdk vs openjdk-11-jdk with same error:
    • sudo apt install libsolr-java=3.6.2+dfsg-11 solr-common=3.6.2+dfsg-11 solr-jetty=3.6.2+dfsg-11
    • sudo apt install openjdk-8-jdk
  • Possibly permission issue similar to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919638

CURRENT SOLUTION

I'm not adding this as an answer as this just abandons using the solr-jetty package which doesn't answer the question or how to get solr-jetty to work. However, after getting solr-jetty to work (see below answer) I've decided to switch to using this solution which uses solr directly.

  • Follow CKAN docs steps 1-4
  • Install solr 6.5.1 instead of ckan's step 5
    • solr >= 6.6.0 has errors with the CKAN schema. Altering the schema is possible but that's even more to maintain / trouble shoot.
    • checkout the ckan wiki and Solr for some instructions
    • copy the solr schema over to the new solr install
  • Continue on with step 6
like image 472
user3366016 Avatar asked Sep 06 '25 02:09

user3366016

1 Answers

Currently the solr-jetty package breaks for this setup. It appears to be a permission and file issue. Here is one way to get it working.

Thanks goes to this PR!

  1. Easiest way for me was to follow all the CKAN source install docs from start to finish including solr (but I did skip datastore as I didn't want it)
    • At this point the site should load but you'll get solr errors

  2. Make the following solr updates:

    1. sudo mkdir /etc/systemd/system/jetty9.service.d

    2. sudo nano /etc/systemd/system/jetty9.service.d/solr.conf and add

      [Service] ReadWritePaths=/var/lib/solr

    3. sudo nano /etc/solr/solr-jetty.xml and replace with the below configuration.
  3. Restart jetty9 sudo service jetty9 restart and you may have to systemctl daemon-reload

Step 3.3 solr-jetty.xml:

<?xml version="1.0"  encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">

<!-- Context configuration file for the Solr web application in Jetty -->

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/solr</Set>
  <Set name="war">/usr/share/solr/web</Set>

  <!-- Set the solr.solr.home system property -->
  <Call name="setProperty" class="java.lang.System">
    <Arg type="String">solr.solr.home</Arg>
    <Arg type="String">/usr/share/solr</Arg>
  </Call>

  <!-- Enable symlinks -->
  <!-- Disabled due to being deprecated
  <Call name="addAliasCheck">
    <Arg>
      <New class="org.eclipse.jetty.server.handler.ContextHandler$ApproveSameSuffixAliases"/>
    </Arg>
  </Call>
  -->
</Configure>

See this issue for additional info.

like image 74
user3366016 Avatar answered Sep 07 '25 21:09

user3366016
Sign in to Comment

Related questions

Basic Authentication is not working in Apache solr 8

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!