Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

marklogic 8 query performance down after inserting large number xml files in my database

I inserted 200000 xml document (approximately Total size 1GB) in my database through MLCP command. Now I want to run below search query against that database (database with default index setup in the admin api) to get all documents.

let $options :=
 <options xmlns="http://marklogic.com/appservices/search">
  <search-option>unfiltered</search-option>
  <term>
    <term-option>case-insensitive</term-option>
  </term>
  <constraint name="Title">
      <range collation="http://marklogic.com/collation/" facet="true">
       <element ns="http://learning.com" name="title" />
      </range>
  </constraint>
  <constraint name="Keywords">
      <range collation="http://marklogic.com/collation/" facet="true">
       <element ns="http://learning.com" name="subjectKeyword" />
      </range>
  </constraint>
  <constraint name="Subjects">
      <range collation="http://marklogic.com/collation/" facet="true">
       <element ns="http://learning.com" name="subjectHeading" />
      </range>
  </constraint>
  <return-results>true</return-results>
  <return-query>true</return-query>
</options>
let $result := search:search("**", $options, 1, 20)
return $result

Range Index:-

      <range-element-index>
      <scalar-type>string</scalar-type>
      <namespace-uri>http://learning.com</namespace-uri>
      <localname>title</localname>
      <collation>http://marklogic.com/collation/</collation>
      <range-value-positions>false</range-value-positions>
      <invalid-values>ignore</invalid-values>
    </range-element-index>
      <range-element-index>
      <scalar-type>string</scalar-type>
      <namespace-uri>http://learning.com</namespace-uri>
      <localname>subjectKeyword</localname>
      <collation>http://marklogic.com/collation/</collation>
      <range-value-positions>false</range-value-positions>
      <invalid-values>ignore</invalid-values>
    </range-element-index>
          <range-element-index>
      <scalar-type>string</scalar-type>
      <namespace-uri>http://learning.com</namespace-uri>
      <localname>subjectHeading</localname>
      <collation>http://marklogic.com/collation/</collation>
      <range-value-positions>false</range-value-positions>
      <invalid-values>ignore</invalid-values>
    </range-element-index>

In each xml document subjectkeyword and title value like be

<lmm:subjectKeyword>anatomy, biology, illustration, cross, section, digestive, human, circulatory, body, small, neck, head, ear, torso, veins, teaching, model, deep, descending, heart, brain, muscles, lungs, diaphragm, c</lmm:subjectKeyword><lmm:title>CORTY_EQ07-014.eps</lmm:title>

But it taking lots of time even query console saying Too many elements to render or Parser Error: Cannot parse result. File Size too large

like image 280
Raj Avatar asked Jan 20 '26 17:01

Raj


1 Answers

I'd also add that if you wanted to fetch all documents (which I wouldn't recommend on a non-trivial database) doing it directly rather than as a wildcarded search is going to be more efficient: fn:doc() (or, as Geert suggests, paginated: fn:doc[1 to 20]

like image 200
mholstege Avatar answered Jan 22 '26 12:01

mholstege