urls and emails in solr search results

Question

I have a solr search returning contextual highlight results showing url's and emails with a space after periods -- like "www. google. com" or "email@google. com" Is there anyway to turn this off, so they display normally? Thanks.

Query:

/solr/core1/select?q=email&start=0&rows=10&sort=score+desc&fq=%28categories%3A%28*8*%29%29&wt=javabin&version=2

highlight section in config:

<searchComponent class="solr.HighlightComponent" name="highlight">
<highlighting>
  <!-- Configure the standard fragmenter -->
  <!-- This could most likely be commented out in the "default" case -->
  <fragmenter name="gap" 
              default="true"
              class="solr.highlight.GapFragmenter">
    <lst name="defaults">
      <int name="hl.fragsize">250</int>
    </lst>
  </fragmenter>

  <!-- A regular-expression-based fragmenter 
       (for sentence extraction) 
    -->
  <fragmenter name="regex" 
              class="solr.highlight.RegexFragmenter">
    <lst name="defaults">
      <!-- slightly smaller fragsizes work better because of slop -->
      <int name="hl.fragsize">70</int>
      <!-- allow 50% slop on fragment sizes -->
      <float name="hl.regex.slop">0.5</float>
      <!-- a basic sentence pattern -->
      <str name="hl.regex.pattern">[-\w ,/\n\&quot;&apos;]{20,200}</str>
    </lst>
  </fragmenter>

  <!-- Configure the standard formatter -->
  <formatter name="html" 
             default="true"
             class="solr.highlight.HtmlFormatter">
    <lst name="defaults">
      <str name="hl.simple.pre"><![CDATA[<b class="highlite">]]></str>
      <str name="hl.simple.post"><![CDATA[</b>]]></str>
    </lst>
  </formatter>

  <!-- Configure the standard encoder -->
  <encoder name="html" 
           class="solr.highlight.HtmlEncoder" />

  <!-- Configure the standard fragListBuilder -->
  <fragListBuilder name="simple" 
                   default="true"
                   class="solr.highlight.SimpleFragListBuilder"/>

  <!-- Configure the single fragListBuilder -->
  <fragListBuilder name="single" 
                   class="solr.highlight.SingleFragListBuilder"/>

  <!-- default tag FragmentsBuilder -->
  <fragmentsBuilder name="default" 
                    default="true"
                    class="solr.highlight.ScoreOrderFragmentsBuilder">
    <!-- 
    <lst name="defaults">
      <str name="hl.multiValuedSeparatorChar">/</str>
    </lst>
    -->
  </fragmentsBuilder>
</highlighting>

Updated schema:

<fieldType name="text" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
  <analyzer type="index">
    <!-- <tokenizer class="solr.WhitespaceTokenizerFactory"/> -->
    <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
    <!-- in this example, we will only use synonyms at query time
    <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
    -->
    <!-- Case insensitive stop word removal.
      add enablePositionIncrements=true in both the index and query
      analyzers to leave a 'gap' for more accurate phrase queries.
    -->
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
    <filter class="solr.PorterStemFilterFactory"/>
  </analyzer>

  <analyzer type="query">
    <!-- <tokenizer class="solr.WhitespaceTokenizerFactory"/> -->
    <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
    <filter class="solr.PorterStemFilterFactory"/>
  </analyzer>
</fieldType>

Fields

<fields>
.....
<field name="tease" type="text" indexed="false" stored="true" />
<field name="text" type="text" indexed="false" stored="true" />
<field name="fulltext" type="text" indexed="true" stored="false" multiValued="true"/>
</fields>

<copyField source="text" dest="fulltext"/>
<copyField source="title" dest="fulltext"/>
<copyField source="tease" dest="fulltext"/>
<defaultSearchField>fulltext</defaultSearchField>

Answer 1

Try using the UAX29URLEmailTokenizerFactory tokenizer, which is just like the standard tokenizer but can recognize email addresses, URLs, domain names and IPv4/IPv6 addresses. An example from the Solr Reference Guide:

In: Visit http://accarol.com/contact.htm?from=external&a=10 or e-mail [email protected]

Out: Visit, http://accarol.com/contact.htm?from=external&a=10, or, email, [email protected]