Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Lucene hot index backup using IndexReader instead of IndexWriter/SnapshotDeletionPolicy

Tags:

java

lucene

Are the following lines of code acceptable to get a hot backup of a lucene index or IndexWriter/SnapshotDeletionPolicy as described in Lucene index backup should be followed?

Directory dir = ...;
IndexReader reader = IndexReader.open(dir);
IndexCommit commit = reader.getIndexCommit();
Collection<String> fileNames = commit.getFileNames();
//copy the files
reader.close();

Even on a locked index you may open a reader on a commit point while a writer may still change the index.

like image 749
yannisf Avatar asked Jun 03 '11 07:06

yannisf


2 Answers

If you have no IndexWriter writing to the index, then the above code is fine.

But an open IndexWriter against the index can easily delete the files referenced/still in use by this IndexReader (for example, when a merge completes) and then your backup will fail.

like image 188
Michael McCandless Avatar answered Sep 17 '22 03:09

Michael McCandless


You need to use a SnapshotDeletionPolicy.

Unless you have an unreleased snapshot, the writer will be free to delete files as it pleases. This will only happen on flush/close, so you might be able to get away with it most of the time, but it won't always work.

Note that the policy is owned by the writer, so if you're trying to somehow use one process to back it up while another process writes, this won't work.

like image 45
Xodarap Avatar answered Sep 21 '22 03:09

Xodarap