How to Backup & Restore ElasticSearch

Question

I have been trying the following methods to Backup & Restore an ElasticSearch cluster from one server to another with little success. I have not used a backup process thus far and I would like to move my whole ElasticSearch cluster from a small 2GB cluster to 15GB cluster. I used the following methods.

1. Using taskrabit/elasticsearch-dump - I was successfully able to export the complete database to backup.json file, however when restoring the backup.json, it gave me the following output. After researching the output further I understood that the bulk input of the plug-in was not fully developed.

    ./bin/elasticdump --all=true --input=/home/user/backup.json --output=http://192.168.0.213:9200/ --type=data
    Thu, 09 Feb 2017 06:43:29 GMT | starting dump
    Thu, 09 Feb 2017 06:43:29 GMT | got 61 objects from source file (offset: 0)
    Thu, 09 Feb 2017 06:43:29 GMT | sent 61 objects to destination elasticsearch, wrote 0
    Thu, 09 Feb 2017 06:43:29 GMT | got 0 objects from source file (offset: 61)
    Thu, 09 Feb 2017 06:43:29 GMT | Total Writes: 0
    Thu, 09 Feb 2017 06:43:29 GMT | dump complete
   

2. Using elasticsearch-tools (es-export-bulk & es-import bulk) I again was able to backup the json successfully. But the import failed once again with an error:

    "statusCode":400,"response":"{"error":{ 
   

I used the examples from es-bulk-export

3. Using the ElasticSearch built-in Snapshot & Restore.

    curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
      "type": "fs",
       "settings": {"location": "/home/shawn/backup", "compress": true}
    }'
   

I believe I'm missing something as the execution gives me the following error. Do I need to create /_snapshot/my_backup? If so how?

{"error":{"root_cause":[
  {"type":"repository_exception",
   "reason":"[my_backup] location [/home/shawn/backup] doesn't match any of the locations specified by path.repo because this setting is empty"
  }],
   "type":"repository_exception","reason":"[my_backup] failed to create repository",
   "caused_by":
       {"type":"creation_exception","reason":"Guice creation errors:\n\n1) Error injecting constructor, RepositoryException[[my_backup] location [/home/shawn/backup] doesn't match any of the locations specified by path.repo because this setting is empty]\n  at org.elasticsearch.repositories.fs.FsRepository.<init>(Unknown Source)\n  while locating org.elasticsearch.repositories.fs.FsRepository\n  while locating org.elasticsearch.repositories.Repository\n\n1 error","caused_by":{"type":"repository_exception","reason":"[my_backup] location [/home/shawn/backup] doesn't match any of the locations specified by path.repo because this setting is empty"}}},"status":500}

Answer 1

You are creating your _snapshot/my_backup just fine, You only need to add line to /etc/elasticsearch/elasticsearch.yml:

path.repo: ["/home/shawn/backup"]

Which is actual location of your snapshot. Then restart Elasticsearch.