why elasticsearch won't run on Ubuntu 14.04?

Question

I'm trying to determine if the elasticsearch instance is running, but it doesn't appear to be:

ubuntu@ubuntu:~$ sudo service elasticsearch status  * elasticsearch is not running ubuntu@ubuntu:~$ sudo service elasticsearch start  * Starting Elasticsearch Server  [ OK ]  ubuntu@ubuntu:~$ sudo service elasticsearch status  * elasticsearch is not running and  ubuntu@ubuntu:~$ sudo /etc/init.d/elasticsearch status  * elasticsearch is not running ubuntu@ubuntu:~$ sudo /etc/init.d/elasticsearch start  * Starting Elasticsearch Server  [ OK ]  ubuntu@ubuntu:~$ sudo /etc/init.d/elasticsearch status  * elasticsearch is not running ubuntu@ubuntu:/etc/elasticsearch# sudo service elasticsearch restart  * Stopping Elasticsearch Server  [ OK ]   * Starting Elasticsearch Server  [ OK ]  ubuntu@ubuntu:/etc/elasticsearch# sudo service elasticsearch status  * elasticsearch is not running 

and

ubuntu@ubuntu:~$ curl -XGET localhost:9200/_nodes/_all/process?pretty curl: (7) Failed to connect to localhost port 9200: Connection refused 

and

ubuntu@ubuntu:/etc/elasticsearch$ sudo netstat -nlp tcp6       0      0 :::9300                 :::*                    LISTEN      4413/java        

UPD

My elasticsearch.log:

[2014-12-03 00:00:02,161][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [_default_] [2014-12-03 00:00:02,617][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic) [2014-12-03 00:00:12,737][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic) [2014-12-03 00:00:17,587][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic) [2014-12-03 00:00:18,842][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic) [2014-12-03 01:00:01,430][INFO ][cluster.metadata         ] [Zero] [logstash-2014.11.25] deleting index [2014-12-03 09:46:57,461][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic) 

Answer 1

Elasticsearch service init script doesn't print any error information on console or log file when it fails to startup, instead it ridiculously shows [OK].

You have to run elaticsearch manually with the same user and same parameters as what the init script does to check what's going wrong. The error message will be printed on console.

On my Ubuntu 14.10 with elasticsearch-1.4.1.deb installed, without any path changed, the command to run elastisearch is:

sudo -u elasticsearch /usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch.pid --default.config=/etc/elasticsearch/elasticsearch.yml --default.path.home=/usr/share/elasticsearch --default.path.logs=/var/log/elasticsearch --default.path.data=/var/lib/elasticsearch --default.path.work=/tmp/elasticsearch --default.path.conf=/etc/elasticsearch 

I just added a line into /etc/init.d/elasticsearch to print out the above command:

# Start Daemon log_daemon_msg "sudo -u $ES_USER $DAEMON $DAEMON_OPTS"    # <-- Add this line start-stop-daemon --start -b --user "$ES_USER" -c "$ES_USER" --pidfile "$PID_FILE" --exec $DAEMON -- $DAEMON_OPTS log_end_msg $? 

Answer 2

The elasticsearch user cannot write the PID file because it has no permissions to create a file in /var/run/:

FileNotFoundException[/var/run/elasticsearch.pid (Keine Berechtigung)] 

The fix: create the directory /var/run/elasticsearch/, change its ownership to elasticsearch:elasticsearch, and change the PID file location to this directory in the init script.

mkdir -p /var/run/elasticsearch chown elasticsearch: /var/run/elasticsearch sed -e 's|^PID_FILE=.*$|PID_FILE=/var/run/$NAME/$NAME.pid|g' -i /etc/init.d/elasticsearch 

Once you get that far, here's the next error you might see:

ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/var/lib/elasticsearch/elasticsearch]] 

Again a resource does not have the correct permissions for the elasticsearch user.

chown -R elasticsearch: /var/lib/elasticsearch 

Not done yet. Now you have to edit /etc/init.d/elasticsearch and remove this line:

test "$START_DAEMON" == true || exit 0 

This line is utter garbage and is guaranteed to cause an exit.

Now it should finally start.