Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Modify default number of Elasticsearch shards

If I have a 15 node cluster, do I have to change the

index.number_of_shards

value on all 15 nodes, and restart them, before the new value comes into effect for new indexes?

like image 387
David H. Avatar asked Aug 27 '15 17:08

David H.


People also ask

How do I reduce number of shards in Elasticsearch?

If you're using time-based index names, for example daily indices for logging, and you don't have enough data, a good way to reduce the number of shards would be to switch to a weekly or a monthly pattern. You can also group old read-only indices., by month, quarter or year.

How do I increase shard limit in Elasticsearch?

Possible solutions from the link above are: - Add more nodes to your Elasticsearch cluster. - Delete old indices if these couldn't be necessary anymore. - Increase the shards limit by nodes.

How many shards should Elasticsearch indexes have?

A good rule-of-thumb is to ensure you keep the number of shards per node below 20 per GB heap it has configured. A node with a 30GB heap should therefore have a maximum of 600 shards, but the further below this limit you can keep it the better.

What is the default shards Elasticsearch?

An Elasticsearch index consists of one or more primary shards. As of Elasticsearch version 7, the current default value for the number of primary shards per index is 1.


1 Answers

That is right changing index.number_of_shards defaults in config file would involve changing the setting on all nodes and then restarting the instance ideally following the guidelines for rolling restarts.

However if that is not an option and if explicitly specifying the number_of_shards in the settings while creating the new index is not ideal then the workaround would be using index templates

Example:

One can create an index_defaults default as below

PUT /_template/index_defaults 
{
  "template": "*", 
  "settings": {
    "number_of_shards": 4
  }
}

This applies the setting specified in index_defaults template to all new indexes.

like image 80
keety Avatar answered Oct 11 '22 14:10

keety