Missing newline for adding with bulk API

Question

I want to add the following file to Elasticsearch using the bulk API:

{"_id":{"date":"01-2007","profile":"Da","dgo":"DGO_E_AIEG","consumerType":"residential"},"value":{"min":120.42509,"minKwh":0.20071,"nbItems":6.0}}

using the command

curl -XPOST -H 'Content-Type: application/json' localhost:9200/_bulk --data-binary Downloads/bob/test.json

but I got the following mistake:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\n]"}],"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\n]"},"status":400}

NB: The file clearly has a empty line at the end

Answer 1

In the docs it says:

NOTE: the final line of data must end with a newline character \n.

There is an example above that of what the document is expected to look like. https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html. Perhaps adding \n at the end of each line would fix the issue.

UPDATE:

There might be something wrong with the way you have placed your data into your JSON file. For example, the following data is in example.json:

{ "index" : { "_index" : "example", "_type" : "doc", "_id" : "1" } }
{ "field1" : "value1" }
<space here>

When running the following curl command, it works:

curl -X POST -H "Content-Type: application/x-ndjson" localhost:9200/_bulk --data-binary "@example.json"

It could be that you're not including something important in your JSON file, or you don't have "@your_file.json", or like the other poster mentioned, you don't have the content-type as application/x-ndjson.

Answer 2

The answer is very simple

{ "index":{  "_index":"schools_gov", "_type":"school", "_id":"1" } }
{ "name":"Model School", "city":"Hyderabad"}
{ "index":{  "_index":"schools_gov", "_type":"school", "_id":"2" } }
{ "name":"Government School", "city":"Pune"}

is not going to work but the below json will work

{ "index":{  "_index":"schools_gov", "_type":"school", "_id":"1" } }
{ "name":"Model School", "city":"Hyderabad"}
{ "index":{  "_index":"schools_gov", "_type":"school", "_id":"2" } } 
{ "name":"Government School", "city":"Pune"}
//Give a new line here. Not '\n' but the actual new line.

The HTTP command would be POST http://localhost:9200/schools_gov/_bulk