<p>There are no examples in the documentation, and I was just wondering what to expect from input I give it.</p>

<p>Before Elasticsearch 0.16 (Lucene 3.1) the standard token filter was "normalizing tokens extracted by standard tokenizer". To be specific, it was removing <code>'s</code> at the end of words and dots in acronyms. So, back then <code>Apple's</code> <code>C.E.O</code> would become <code>Apple</code> <code>CEO</code> after passing the standard filter. Starting with Elasticsearch 0.16 (Lucene 3.1) the standard token filter does nothing (at least at the moment). It simply passes tokens to the next filter in the chain. </p>

What exactly does the Standard tokenfilter do in Elasticsearch?

1 Answers

Before Elasticsearch 0.16 (Lucene 3.1) the standard token filter was "normalizing tokens extracted by standard tokenizer". To be specific, it was removing 's at the end of words and dots in acronyms. So, back then Apple's C.E.O would become Apple CEO after passing the standard filter. Starting with Elasticsearch 0.16 (Lucene 3.1) the standard token filter does nothing (at least at the moment). It simply passes tokens to the next filter in the chain.