Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ElasticSearch NEST: Create an index through ElasticClient by specifying json

We allow the client to define custom analyzers at the time they create an index. We would prefer to specify this in json to provide maximum flexibility and understandability via the underlying ElasticSearch documentation.

I would like to create an index using an arbitrary description of analyzers, mappers, etc., defined in a json string. Using sense, my command is

PUT /my_index
{
    "settings": 
    {
        "analysis": 
        {
            "char_filter" : 
            {
                "my_mapping" : 
                {
                    "type" : "mapping",
                    "mappings" : [".=>,", "'=>,"]
                }
            },
            "analyzer": 
            {
                "my_analyzer": 
                {
                    "type":         "custom",
                    "tokenizer":    "standard",
                    "filter":       ["lowercase" ],
                    "char_filter" : ["my_mapping"]
                }
            }
         }
      }
   }
}

Ideally, my code would look something like

string json = RetrieveJson();
ElasticSearchClient client = InitializeClient();
client.CreateIndexUsingJson( json ); // this is the syntax I can't figure out

The post here attempts to do this by instantiating an IndexSettings then calling Add( "analysis", json ), but Add is not a function on the ElasticSearch library version I'm using.

The options I can imagine include:

  1. Somehow using the ElasticClient.Raw.IndicesCreatePost or something analogous
  2. Deserializing the json string into an IndexSettings object via IndexSettingsConverter.ReadJson(), and then applying that through ElasticClient.CreateIndex(ICreateIndexRequest)

Both of these mechanisms have very scant documentation.

I'm absolutely trying to avoid the lambda function versions of CreateIndex, since it would be miserable to translate the user's json into lamdba expressions, only to immediately translate them back into json deep in NEST.

Other options or concrete examples of #1 or #2 above are very much appreciated, as is a recommended approach to solving this problem.

like image 951
mcating Avatar asked Mar 13 '15 06:03

mcating


People also ask

How do I create an index in Kibana?

Open the main menu, then click to Stack Management > Index Patterns. Click Create index pattern. Start typing in the Index pattern field, and Kibana looks for the names of indices, data streams, and aliases that match your input.


1 Answers

Easiest solution was an implementation of Option #1 from the original question.

public void CreateIndex(string indexName, string json)
{
    ElasticClient client = GetClient();
    var response = _client.Raw.IndicesCreatePost(indexName, json);
    if (!response.Success || response.HttpStatusCode != 200)
    {
        throw new ElasticsearchServerException(response.ServerError);
    }
}

After tinkering around with converters and JsonReaders and JsonSerializers, I found that the IndexSettingsConverter didn't seem to properly deserialize arbitrary settings json into a valid IndexSettings object. Sensing a rabbit hole, I took Manolis' suggestion and figured out how to apply the arbitrary json directly against the ElasticClient.IElasticsearchClient to avoid having to reverse-engineer security and connection details.

Painful effort to come to this conclusion, and completely impossible without working through a whole lot of undocumented NEST code.

like image 76
mcating Avatar answered Nov 06 '22 17:11

mcating