Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to set fielddata=true in kibana

Tags:

kibana

I am new to Kibana, have data loaded into Elastic 5.0.0-alpha3 and am using Kibana 5.0.0-alpha3 to Visualise. I can display some numeric fields as histograms but when I want to use text fields I get:

Visualize: Fielddata is disabled on text fields by default. Set fielddata=true on [publisher] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. 

I am warned that the data (names of Publishers) may have been analyzed into subfields, but I'd like to display anyway.

How do I set fielddata=true?

EDIT: recent issues on Kibana github suggest that this is a new feature in 5.0.0 and is still awaiting an answer!

EDIT (following @Val's answer, and asking for Elastic newbie help, and hoping others will find it useful). The ingest script was:

fs = require('fs')  var elasticsearch = require('elasticsearch'); var client = new elasticsearch.Client({  host: 'localhost:9200',  log: 'trace' });  fs.readFile('/Users/pm286/workspace/cmdev/getpapers/20160602/crossref_results.json', (err, data) => {   if (err) throw err;    document = JSON.parse(data)   document = JSON.parse(data)    for(i=0;i<document.length;i++) {       client.create({           index: 'index',           type: 'type',           body: document[i]           })       }   }); 

How do I include @Val's approach in this?

like image 982
peter.murray.rust Avatar asked Jul 01 '16 12:07

peter.murray.rust


People also ask

How do you set Fielddata to true elastic?

Set fielddata=true on [event. dataset] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.: Check the Elasticsearch Monitoring cluster network connection or the load level of the nodes.

What is Fielddata true in Elasticsearch?

Fielddata is an in-memory data structure used by text fields for the same purpose. Since it uses a lot of heap size it is disabled by default.

What is keyword field in Elasticsearch?

The keyword family includes the following field types: keyword , which is used for structured content such as IDs, email addresses, hostnames, status codes, zip codes, or tags. constant_keyword for keyword fields that always contain the same value.


1 Answers

In your ES mapping, you need to set fielddata:true in your publisher field:

PUT your_index/_mapping/your_type {    "your_type": {       "properties": {         "publisher": {           "type": "text",           "fielddata": true         }       }    } } 

You'll need to reindex your data after making this change, but afterwards Kibana won't complain anymore.

UPDATE

You can either execute the above query in the Sense UI or through curl

curl -XPUT http://localhost:9200/index -d '{   "mappings": {     "type": {       "properties": {         "publisher": {           "type": "text",           "fielddata": true         }       }     }   } }' 

Or you can also execute it in your Javascript file just before creating your document:

client.indices.create({   index: 'index',   body: {       "mappings": {         "type": {           "properties": {             "publisher": {               "type": "text",               "fielddata": true             }           }         }       }     } }); 
like image 52
Val Avatar answered Sep 17 '22 15:09

Val