How to get the unique results from Lucene index?

Question

I am trying to search from lucene index . I want to get the unique results but its returning the duplicate results also. I searched on google and found it can be done with the help of a collector. How can I achieve this?

I am using the following code:

File outputdir= new File("path upto lucene directory");
Directory directory = FSDirectory.open(outputdir);
IndexSearcher= new IndexSearcher(directory,true);

QueryParser queryparser = new QueryParser(Version.LUCENE_36, "keyword", new StandardAnalyzer(Version.LUCENE_36));

Query query = queryparser.parse("central");

topdocs = indexSearcher.search(query, maxhits);
ScoreDoc[] score = topdocs.scoreDocs;
int length = score.length;

Answer 1

Are you indexing content before each search ?

If so, I suggest you to separate indexing code and searching code because if you launch this script several times without deleting the index folder Lucene doesn't overwrite the index but add again the content to the index. I think this is why you get duplicates results.

Answer 2

You should have a field named for example "duplicate" and set the value to "true" on indexing time when it already has a duplicate in the index.

So you can search for

Query query = queryparser.parse("central -duplicate:true");