Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a HTML analyzer/tokenizer for Lucene?

Tags:

lucene

I wanted to index text from html, in Lucene, what is the best way to achieve this ?
Is there any good Contrib module that can do this in Lucene ?

EDIT
Finally ended up using Jericho Parser. It doesn't create DOM and is easy to use.

like image 861
Shrinath Avatar asked Mar 11 '11 10:03

Shrinath


3 Answers

I'm assuming that you don't actually want to index the HTML tags. If that's the case, you can first extract text from HTML using Apache Tika. Then you can index the text in Lucene.

like image 112
bajafresh4life Avatar answered Nov 17 '22 15:11

bajafresh4life


I would recommend using Jsoup HTML parser to extract the text and then use Lucene. It worked well for me.

like image 20
remo Avatar answered Nov 17 '22 17:11

remo


You might also want to take a look at /Lucene-3.0.3/src/demo which has an HTML parser example.

like image 2
Gene Golovchinsky Avatar answered Nov 17 '22 17:11

Gene Golovchinsky