Auto completion search with Solr using NGrams

Question

I'm working on auto completion search with Solr using EdgeNGrams. If the user is searching for names of employees, then auto completion should be applied. That is, I want the results to be like a Google search. It's working fine for some searches.

File schema.xml:

<fieldType name="edgytext" class="solr.TextField" positionIncrementGap="100" omitNorms="true">
    <analyzer type="index">
        <tokenizer class="solr.KeywordTokenizerFactory" />
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
    </analyzer>
    <analyzer type="query">
        <tokenizer class="solr.KeywordTokenizerFactory" />
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="15" side="front" />
    </analyzer>

<field name="title" type="edgytext" indexed="true" stored="true" omitNorms="true" omitTermFreqAndPositions="true"/>
<field name="empname" type="edgytext" indexed="true" stored="true" omitNorms="true" omitTermFreqAndPositions="true" />

<field name="autocomplete_text" type="edgytext" indexed="true" stored="false"  multiValued="true" omitNorms="true" omitTermFreqAndPositions="false" />
<copyField source="empname" dest="autocomplete_text"/>

<copyField source="title" dest="autocomplete_text"/>

---

 http://local:8080/test/suggest/?q=michael

Result:

<?xml version="1.0" encoding="UTF-8" ?>
<response>
    <lst name="responseHeader">
        <int name="status">0</int>
        <int name="QTime">1</int>
    </lst>
    <result name="response" numFound="0" start="0" />
    <lst name="spellcheck">
        <lst name="suggestions">
            <lst name="michael">
                <int name="numFound">9</int>
                <int name="startOffset">0</int>
                <int name="endOffset">7</int>
                <arr name="suggestion">
                    <str>michael bolton</str>
                    <str>michael foret</str>
                    <str>michael houser</str>
                    <str>michael o'brien</str>
                    <str>michael penn</str>
                    <str>michael row your boat ashore</str>
                    <str>michael tilson thomas</str>
                    <str>michael w. smith</str>
                    <str>michael w. smith featuring andrae crouch</str>
                </arr>
            </lst>
            <str name="collation">michael bolton</str>
        </lst>
    </lst>
</response>

It's working fine for me. When I search with michael f

http:// local:8080/test/suggest/?q=michael f

I get a response like:

<?xml version="1.0" encoding="UTF-8" ?>
<response>
    <lst name="responseHeader">
        <int name="status">0</int>
        <int name="QTime">1</int>
    </lst>
    <result name="response" numFound="0" start="0" />
    <lst name="spellcheck">
        <lst name="suggestions">
            <lst name="michael">
                <int name="numFound">9</int>
                <int name="startOffset">0</int>
                <int name="endOffset">7</int>
                <arr name="suggestion">
                    <str>michael bolton</str>
                    <str>michael foret</str>
                    <str>michael houser</str>
                    <str>michael o'brien</str>
                    <str>michael penn</str>
                    <str>michael row your boat ashore</str>
                    <str>michael tilson thomas</str>
                    <str>michael w. smith</str>
                    <str>michael w. smith featuring andrae crouch</str>
                </arr>
            </lst>
            <lst name="f">
                <int name="numFound">10</int>
                <int name="startOffset">8</int>
                <int name="endOffset">9</int>
                <arr name="suggestion">
                    <str>f**k the facts</str>
                    <str>fairest lord jesus</str>
                    <str>fatboy slim</str>
                    <str>ffh</str>
                    <str>fiona apple</str>
                    <str>foo fighters</str>
                    <str>frank sinatra</str>
                    <str>frans bauer</str>
                    <str>franz ferdinand</str>
                    <str>françois rauber</str>
                </arr>
            </lst>
            <str name="collation">michael bolton f**k the facts</str>
        </lst>
    </lst>
</response>.

When I search with michael f then, I should get michael foret only. Data coming starts with f. Is there anything wrong in my configuration settings in Solr?

Answer 1

I wrote [old link] about different ways to make auto-suggestions with Solr and some questions you should ask yourself in order to make the right choice. Briefly, the out-of-the-box ways are:

• Facet prefix
• NGrams
• TermsComponent
• Suggester

They all have advantages and limitations at the same time, I'd suggest you to read the article.

If you are looking for a complete and flexible solution, which requires some more work, you can have a look at this article as well.

If you already decided to use NGrams, given your examples you can index your employees using EdgeNGramFilterFactory with minGramSize 1, then search on that field to make auto-suggestions. For the client part you need to use some javascript.