Lucene can't find documents after update

Question

It seems that whenever I update an existing document in the index (same behavior for delete / add), it can't be found with a TermQuery. Here's a short snippet:

iw = new IndexWriter(directory, config);

Document doc = new Document();
doc.add(new StringField("string", "a", Store.YES));
doc.add(new IntField("int", 1, Store.YES));

iw.addDocument(doc);

Query query = new TermQuery(new Term("string","a"));

Document[] hits = search(query);
doc = hits[0];
print(doc);

doc.removeField("int");
doc.add(new IntField("int", 2, Store.YES));

iw.updateDocument(new Term("string","a"), doc);

hits = search(query);
System.out.println(hits.length);
System.out.println("_________________");

for(Document hit : search(new MatchAllDocsQuery())){
    print(hit);
}

This produces the following console output:

stored,indexed,tokenized,omitNorms,indexOptions=DOCS_ONLY<string:a>
stored<int:1>
________________
0
_________________
stored,indexed,tokenized,omitNorms,indexOptions=DOCS_ONLY<string:a>
stored<int:2>
________________

It seems that after the update, the document (rather the new document) in the index and gets returned by the MatchAllDocsQuery, but can't be found by a TermQuery.

Full sample code available at http://pastebin.com/sP2Vav9v

Also, this only happens (second search not working) when the StringField value contains special characters (e.g. file:/F:/).

Answer 1

The code which you have referenced in pastebin doesn't find anything because your StringField is nothing but a stopword (a). Replacing a with something which is not a stopword (e.g. ax) makes both searches to return 1 doc.

You would also achieve the correct result if you were to construct StandardAnalyzer with empty stopword set (CharArraySet.EMPTY_SET) yet still using a for StringField. This wouldn't work for file:/F:/ though.

However, the best solution is this case would be to replace StandardAnalyzer with KeywordAnalyzer.