Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to add an elasticsearch index during docker build

I use the official elasticsearch docker image and wonder how can I include also during building a custom index, so that the index is already there when I start the container.

My attempt was to add the following line to my dockerfile:

RUN curl -XPUT 'http://127.0.0.1:9200/myindex' -d @index.json 

I get the following error:

0curl: (7) Failed to connect to 127.0.0.1 port 9200: Connection refused 

Can I reach elasticsearch during build with such an API call or is there a complete different way to implement that?

like image 926
crisscross Avatar asked Feb 20 '16 17:02

crisscross


People also ask

How do I run Elasticsearch in a docker container?

Create a file named elasticsearch.yml in the same directory as the Dockerfile, with this content: Also, to get logging to work with docker we should add a simple logging.yml file: To add these files to the container we add the following to the Dockerfile: What’s left now is to actually make the container run Elasticsearch at startup.

Where can I find the source files for Elasticsearch?

Elasticsearch is also available as Docker images. The images use centos:8 as the base image. A list of all published Docker images and tags is available at www.docker.elastic.co. The source files are in Github. This package contains both free and subscription features. Start a 30-day trial to try out all of the features.

How to create an index in Elasticsearch cluster?

The syntax for creating a new index in Elasticsearch cluster is: To create an index, all you have to do is pass the index name without other parameters, which creates an index using default settings. You can also specify various features of the index, such as in the index body:

How do I create a keystore file in Elasticsearch?

By default, Elasticsearch will auto-generate a keystore file for secure settings. This file is obfuscated but not encrypted. To encrypt your secure settings with a password and have them persist outside the container, use a docker run command to manually create the keystore instead.


1 Answers

I've had a similar problem.

I wanted to create a docker container with preloaded data (via some scripts and json files in the repo). The data inside elasticsearch was not going to change during the execution and I wanted as few build steps as possible (ideally only docker-compose up -d).

One option would be to do it manually once, and store the elasticsearch data folder (with a docker volume) in the repository. But then I would have had duplicate data and I would have to check in manually a new version of the data folder every time the data changes.

The solution

  1. Make elasticsearch write data to a folder that is not declared as a volume in elasticsearchs' official dockerfile.

RUN mkdir /data && chown -R elasticsearch:elasticsearch /data && echo 'es.path.data: /data' >> config/elasticsearch.yml && echo 'path.data: /data' >> config/elasticsearch.yml

(the folder needs to be created with the right permissions)

  1. Download wait-for-it

ADD https://raw.githubusercontent.com/vishnubob/wait-for-it/e1f115e4ca285c3c24e847c4dd4be955e0ed51c2/wait-for-it.sh /utils/wait-for-it.sh

This script will wait until elasticsearch is up to run our insert commands.

  1. Insert data into elasticsearch

RUN /docker-entrypoint.sh elasticsearch -p /tmp/epid & /bin/bash /utils/wait-for-it.sh -t 0 localhost:9200 -- path/to/insert/script.sh; kill $(cat /tmp/epid) && wait $(cat /tmp/epid); exit 0;

This command starts elasticsearch during the build process, inserts data and takes it down in one RUN command. The container is left as it was except for elasticsearch's data folder which has been properly initialized now.

Summary

FROM elasticsearch  RUN mkdir /data && chown -R elasticsearch:elasticsearch /data && echo 'es.path.data: /data' >> config/elasticsearch.yml && echo 'path.data: /data' >> config/elasticsearch.yml  ADD https://raw.githubusercontent.com/vishnubob/wait-for-it/e1f115e4ca285c3c24e847c4dd4be955e0ed51c2/wait-for-it.sh /utils/wait-for-it.sh  # Copy the files you may need and your insert script  RUN /docker-entrypoint.sh elasticsearch -p /tmp/epid & /bin/bash /utils/wait-for-it.sh -t 0 localhost:9200 -- path/to/insert/script.sh; kill $(cat /tmp/epid) && wait $(cat /tmp/epid); exit 0; 

And that's it! When you run this image, the database will have preloaded data, indexes, etc...

like image 163
Erpheus Avatar answered Oct 04 '22 01:10

Erpheus