Lucene standard analyzer split on period

Question

How do I make Lucene's Standard Analyzer tokenize on the'.' char? For eg., on querying for "B" I need it to return the B in "A.B.C" as the result. I need to treat numbers the way the standard analyzer treats it, and hence the Simple analyzer is not sufficient. It would be perfect if I could just specify to the standard Analyzer to tokenize on the'.' char too. If I had to write my own tokenizer with just this small extension, how would I go about it? Thanks, Nacha

Answer 1

I believe the easiest is to create your own Analyzer. which will get tokens from StandardAnalyzer as input and further split tokens on dots, keeping dotless tokens intact.

The package summary gives some advice on how to do this. This blog post seems very relevant, but uses an old version of Lucene, so you will probably need to tweak it. Also, see the Lucene FAQ.