Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

limit number of thread in ThreadPool while creating TransportClient in elasticsearch

I'm creating a TransportClient instance in elasticsearch. Below is the code for the same. The issue is I'm trying to lower the number of threads spawned with the threadpool that TransportClient initiates. But what ever settings I use my elasticsearch always initialing the threadpool with 12 threads. Please let me know how to configure the same to get the desirable threads.

public static TransportClient getTransportClient(String ip, int port) {

    ImmutableSettings.Builder settings = ImmutableSettings
            .settingsBuilder();
    settings.put("cluster.name", "elasticsearch");
    settings.put("threadpool.bulk.type",  "fixed");
    settings.put("threadpool.bulk.size" ,5);
    settings.put("threadpool.bulk.queue_size", 5);
    settings.put("threadpool.index.type" , "fixed");
    settings.put("threadpool.index.size" , 5);
    settings.put("threadpool.index.queue_size" , 10);
    settings.put("threadpool.search.type",  "fixed");
    settings.put("threadpool.search.size" ,5);
    settings.put("threadpool.search.queue_size", 5);

    settings.build();

    TransportClient instance = new TransportClient(settings)
            .addTransportAddress(new InetSocketTransportAddress(ip, port));

    return instance;
}
like image 873
bagui Avatar asked Jan 14 '15 01:01

bagui


People also ask

What is ThreadPool in Elasticsearch?

Elasticsearch ThreadPool Test. Thread pools are a collection of threads which are made available to perform specific tasks in the Elasticsearch cluster. A single Elasticsearch node holds multiple thread pools for different operations such as search, indexing, bulk operations, and more.

Is elasticsearch multi threaded?

You can use multi thread, this is exactly why elasticsearch is good for: parallelism. An elasticsearch index, is composed of shards, this is the physical storage of your data. Shards can be on the same node or not (better).

What is thread pool size?

The size of a thread pool is the number of threads kept in reserve for executing tasks. It is usually a tunable parameter of the application, adjusted to optimize program performance. Deciding the optimal thread pool size is crucial to optimize performance.


1 Answers

Try

Settings settings = ImmutableSettings.settingsBuilder()
                    .put("transport.netty.workerCount",NUM_THREADS)
                    .build();

Credit to JanuZ, taken from http://www.lucidelectricdreams.com/2013/11/reducing-number-of-threads-created-by.html

like image 106
dux2 Avatar answered Oct 14 '22 13:10

dux2