mergeFactor usage in solr 4.0

Question

I thought mergeFactor could be think as max number of in segment, but when it set to 10, I got 16 segments after index, and merge happened when number of in segment exceed 15 and 20. I am really confused about the mean of mergeFactor.

Here is my config

<ramBufferSizeMB>32</ramBufferSizeMB>
<maxBufferedDocs>100000</maxBufferedDocs>
<mergeFactor>10</mergeFactor>

Answer 1

From the Solr Wiki - SolrPerformanceFactors - Merge Factor documentation

The mergeFactor roughly determines the number of segments. The mergeFactor value tells Lucene how many segments of equal size to build before merging them into a single segment. It can be thought of as the base of a number system.

It is not an exact number, but a guideline for how the index is managed on disk. Here is some additional good guidance on how what different settings of this value mean to the performance of your index:

mergeFactor Tradeoffs

High value merge factor (e.g., 25):

• Pro: Generally improves indexing speed
• Con: Less frequent merges, resulting in a collection with more index files which may slow searching

Low value merge factor (e.g., 2):

• Pro: Smaller number of index files, which speeds up searching.
• Con: More segment merges slow down indexing.

You can also reference the mergeFactor documentation on the Lucidworks site for some more details about how the mergeFcator works and can impact your index performance. But I think the key here is this paragraph:

If creating a new segment would cause the number of lowest-level segments to exceed the mergeFactor value, then all those segments are merged together to form a single large segment. Thus, if the merge factor is ten, each merge results in the creation of a single segment that is roughly ten times larger than each of its ten constituents. When there are mergeFactor settings for these larger segments, then they in turn are merged into an even larger single segment. This process can continue indefinitely.