I am setting a Bulk request to ElasticSearch and specifying the shard to route to.
But when I run it, the documents get sent to different shards.
Is this a bug in ElasticSEarch bulk? it works when I just index a single document. It works when I search. But not when I do a bulk import.
To reproduce:
curl -XPOST 'http://192.168.1.115:9200/_bulk?routing=a' -d '
{ "index" : { "_index" : "articles", "_type" : "article", "_id" : "1" } }
{ "title" : "value1" }
{ "delete" : { "_index" : "articles", "_type" : "article", "_id" : "2" } }
{ "create" : { "_index" : "articles", "_type" : "article", "_id" : "3" } }
{ "title" : "value3" }
{ "update" : {"_id" : "1", "_type" : "article", "_index" : "index1"} }
{ "doc" : {"field2" : "value2"} }'
Routing is the process of determining which shard that document will reside in. Because Elasticsearch tries hard to make defaults work for 90% of users, routing is handled automatically. For most users, it doesn't matter where a document is stored.
The Elastic platform includes ElasticSearch, which is a Lucene-based, multi-tenant capable, and distributed search and analytics engine. The ElasticSearch Bulk Insert step sends one or more batches of records to an ElasticSearch server for indexing.
Bulk Index Tool. When indexing is enabled and operating correctly, an indexable object is indexed as soon as it is created. However, there are times when you need to index large amounts of data at one time. You can use the Bulk Index Tool to load Windchill Index Search libraries and their objects: •
You submit a bulk request to automatically generate bulk quotes or orders. Each action set in a bulk request generates one bulk quote or order, unless an exception occurs. The contacts, services, and actions in the action set determine the line items in the bulk quote or order.
So adding the "routing" parameter to the end of the URL doesn't work.
I need to add the "_routing" field to the actual document fields to specify which shard it will go to.
Very unintuitive, and I wish ElasticSearch would've documented this! Sometimes I wish I just chose Solr :*(
Hope this helps anyone else looking for this in the future
curl -XPOST 'http://192.168.1.115:9200/_bulk?routing=a' -d '
{ "index" : { "_index" : "articles", "_type" : "article", "_id" : "1", "_routing" : "b"} }
{ "title" : "value1" }
{ "delete" : { "_index" : "articles", "_type" : "article", "_id" : "2", "_routing" : "b" } }
{ "create" : { "_index" : "articles", "_type" : "article", "_id" : "3", "_routing" : "b" } }
{ "title" : "value3" }
{ "update" : {"_id" : "1", "_type" : "article", "_index" : "index1", "_routing" : "b"} }
{ "doc" : {"field2" : "value2"} }'
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With