Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why doesn't routing work with ElasticSearch Bulk API?

I am setting a Bulk request to ElasticSearch and specifying the shard to route to.

But when I run it, the documents get sent to different shards.

Is this a bug in ElasticSEarch bulk? it works when I just index a single document. It works when I search. But not when I do a bulk import.

To reproduce:

curl -XPOST 'http://192.168.1.115:9200/_bulk?routing=a' -d '
{ "index" : { "_index" : "articles", "_type" : "article", "_id" : "1" } }
{ "title" : "value1" }
{ "delete" : { "_index" : "articles", "_type" : "article", "_id" : "2" } }
{ "create" : { "_index" : "articles", "_type" : "article", "_id" : "3" } }
{ "title" : "value3" }
{ "update" : {"_id" : "1", "_type" : "article", "_index" : "index1"} }
{ "doc" : {"field2" : "value2"} }'
like image 580
Henley Avatar asked Nov 02 '13 18:11

Henley


People also ask

What is routing in ElasticSearch?

Routing is the process of determining which shard that document will reside in. Because Elasticsearch tries hard to make defaults work for 90% of users, routing is handled automatically. For most users, it doesn't matter where a document is stored.

What is bulk insert in ElasticSearch?

The Elastic platform includes ElasticSearch, which is a Lucene-based, multi-tenant capable, and distributed search and analytics engine. The ElasticSearch Bulk Insert step sends one or more batches of records to an ElasticSearch server for indexing.

What is bulk index?

Bulk Index Tool. When indexing is enabled and operating correctly, an indexable object is indexed as soon as it is created. However, there are times when you need to index large amounts of data at one time. You can use the Bulk Index Tool to load Windchill Index Search libraries and their objects: •

What is bulk request?

You submit a bulk request to automatically generate bulk quotes or orders. Each action set in a bulk request generates one bulk quote or order, unless an exception occurs. The contacts, services, and actions in the action set determine the line items in the bulk quote or order.


1 Answers

So adding the "routing" parameter to the end of the URL doesn't work.

I need to add the "_routing" field to the actual document fields to specify which shard it will go to.

Very unintuitive, and I wish ElasticSearch would've documented this! Sometimes I wish I just chose Solr :*(

Hope this helps anyone else looking for this in the future

curl -XPOST 'http://192.168.1.115:9200/_bulk?routing=a' -d '
{ "index" : { "_index" : "articles", "_type" : "article", "_id" : "1", "_routing" : "b"} }
{ "title" : "value1" }
{ "delete" : { "_index" : "articles", "_type" : "article", "_id" : "2", "_routing" : "b" } }
{ "create" : { "_index" : "articles", "_type" : "article", "_id" : "3", "_routing" : "b" } }
{ "title" : "value3" }
{ "update" : {"_id" : "1", "_type" : "article", "_index" : "index1", "_routing" : "b"} }
{ "doc" : {"field2" : "value2"} }'
like image 161
Henley Avatar answered Oct 12 '22 21:10

Henley