Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dump all documents of Elasticsearch

Is there any way to create a dump file that contains all the data of an index among with its settings and mappings?

A Similar way as mongoDB does with mongodump
or as in Solr its data folder is copied to a backup location.

Cheers!

like image 907
Evan P Avatar asked Oct 08 '13 08:10

Evan P


People also ask

How do I get all Elasticsearch documents?

You can use cURL in a UNIX terminal or Windows command prompt, the Kibana Console UI, or any one of the various low-level clients available to make an API call to get all of the documents in an Elasticsearch index. All of these methods use a variation of the GET request to search the index.

How do I get documents out of Elasticsearch?

You use GET to retrieve a document and its source or stored fields from a particular index. Use HEAD to verify that a document exists. You can use the _source resource retrieve just the document source or verify that it exists.

How do I dump an Elasticsearch database?

Here are three popular methods, you use to export files from Elasticsearch to any desired warehouse or platform of your choice: Elasticsearch Export: Using Logstash-Input-Elasticsearch Plugin. Elasticsearch Export: Using Elasticsearch Dump. Elasticsearch Export: Using Python Pandas.

How do I dump Elasticsearch data to JSON?

Because Elasticsearch Dev Tools and standard Elasticsearch search API limit the number of records to 10,000, the simplest way to export all of the index data as JSON file is to use the elasticsearch-dump tool: https://github.com/elasticsearch-dump/elasticsearch-dump.


2 Answers

Here's a new tool we've been working on for exactly this purpose https://github.com/taskrabbit/elasticsearch-dump. You can export indices into/out of JSON files, or from one cluster to another.

like image 90
Evan Avatar answered Oct 29 '22 00:10

Evan


Elasticsearch supports a snapshot function out of the box:

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

like image 45
Andreas Neumann Avatar answered Oct 29 '22 01:10

Andreas Neumann