[MarkLogic Dev General] RE: database update time

Tue Feb 27 15:48:29 PST 2007

You can create range indexes on properties:  both server-maintained
properties and the ones you add on your own.

I created a range index on the prop:last-modified property with the
following settings:

scalar type: dateTime

namespace uri: http://marklogic.com/xdmp/property

localname: last-modified

I then executed the following query:

xdmp:query-trace(true()),

  (for $t in xdmp:document-properties()//prop:last-modified

   order by $t descending

   return <node uri="{base-uri
($t)}">{xs:dateTime(data($t))}</node>)[1],

xdmp:query-meters()

It returned:

<node uri="new2.xml">2007-06-14T10:07:55-07:00</node>
<qm:query-meters
xsi:schemaLocation="http://marklogic.com/xdmp/query-meters
query-meters.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:qm="http://marklogic.com/xdmp/query-meters">
  <qm:elapsed-time>PT0.174421S</qm:elapsed-time>
  <qm:requests>1</qm:requests>
  <qm:list-cache-hits>12</qm:list-cache-hits>
  <qm:list-cache-misses>0</qm:list-cache-misses>
  <qm:in-memory-list-hits>0</qm:in-memory-list-hits>
  <qm:expanded-tree-cache-hits>4</qm:expanded-tree-cache-hits>
  <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>
  <qm:compressed-tree-cache-hits>0</qm:compressed-tree-cache-hits>
  <qm:compressed-tree-cache-misses>0</qm:compressed-tree-cache-misses>

<qm:in-memory-compressed-tree-hits>0</qm:in-memory-compressed-tree-hits>
  <qm:value-cache-hits>2</qm:value-cache-hits>
  <qm:value-cache-misses>2</qm:value-cache-misses>
  <qm:regexp-cache-hits>0</qm:regexp-cache-hits>
  <qm:regexp-cache-misses>0</qm:regexp-cache-misses>
  <qm:link-cache-hits>0</qm:link-cache-hits>
  <qm:link-cache-misses>0</qm:link-cache-misses>
  <qm:fragments-added>0</qm:fragments-added>
  <qm:fragments-deleted>0</qm:fragments-deleted>
  <qm:fs-program-cache-hits>1</qm:fs-program-cache-hits>
  <qm:fs-program-cache-misses>0</qm:fs-program-cache-misses>
  <qm:db-program-cache-hits>0</qm:db-program-cache-hits>
  <qm:db-program-cache-misses>0</qm:db-program-cache-misses>

<qm:fs-main-module-sequence-cache-hits>0</qm:fs-main-module-sequence-cac
he-hits>

<qm:fs-main-module-sequence-cache-misses>0</qm:fs-main-module-sequence-c
ache-misses>

<qm:db-main-module-sequence-cache-hits>0</qm:db-main-module-sequence-cac
he-hits>

<qm:db-main-module-sequence-cache-misses>0</qm:db-main-module-sequence-c
ache-misses>
  <qm:fs-library-module-cache-hits>0</qm:fs-library-module-cache-hits>

<qm:fs-library-module-cache-misses>0</qm:fs-library-module-cache-misses>
  <qm:db-library-module-cache-hits>0</qm:db-library-module-cache-hits>

<qm:db-library-module-cache-misses>0</qm:db-library-module-cache-misses>
  <qm:fragments>
    <qm:fragment>
      <qm:root
xmlns="http://marklogic.com/xdmp/property">properties</qm:root>
      <qm:expanded-tree-cache-hits>4</qm:expanded-tree-cache-hits>
      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>
    </qm:fragment>
  </qm:fragments>
  <qm:documents>
    <qm:document>
      <qm:uri>new2.xml</qm:uri>
      <qm:expanded-tree-cache-hits>1</qm:expanded-tree-cache-hits>
      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>
    </qm:document>
    <qm:document>
      <qm:uri>new4.xml</qm:uri>
      <qm:expanded-tree-cache-hits>1</qm:expanded-tree-cache-hits>
      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>
    </qm:document>
    <qm:document>
      <qm:uri>new6.xml</qm:uri>
      <qm:expanded-tree-cache-hits>1</qm:expanded-tree-cache-hits>
      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>
    </qm:document>
    <qm:document>
      <qm:uri>new8.xml</qm:uri>
      <qm:expanded-tree-cache-hits>1</qm:expanded-tree-cache-hits>
      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>
    </qm:document>
  </qm:documents>
</qm:query-meters>

We can see that the evaluator actually fetched 4 fragments from disk
while processing the results.  It turns out there are over 10000
fragments in this query.  To see that, we can look at the query-trace
results:

2007-06-14 15:51:41.546 Info: Docs: line 3:
xdmp:eval("xdmp:query-trace(true()),&#13;&#10;  (for $t in
xdmp:document-properties()//prop:last-modified&#13;&#10;  order by $t
descending&#13;&#10;   return <node uri=""{base-uri
($t)}"">{xs:dateTime(data($t))}</node>)[1],&#13;&#10;xdmp:query-meters()
&#13;&#10;")

2007-06-14 15:51:41.546 Info: Docs: line 3: Analyzing path for $t:
xdmp:document-properties()/descendant::prop:last-modified

2007-06-14 15:51:41.546 Info: Docs: line 3: Step 1 is searchable:
xdmp:document-properties()

2007-06-14 15:51:41.548 Info: Docs: line 3: Step 2 is searchable:
descendant::prop:last-modified

2007-06-14 15:51:41.548 Info: Docs: line 3: Path is fully searchable.

2007-06-14 15:51:41.548 Info: Docs: line 3: Gathering constraints.

2007-06-14 15:51:41.548 Info: Docs: line 3: Step 2 test contributed 1
constraint: prop:last-modified

2007-06-14 15:51:41.548 Info: Docs: line 3: Order by clause contributed
1 range ordering constraint for $t: order by $t descending

2007-06-14 15:51:41.548 Info: Docs: line 3: Executing search.

2007-06-14 15:51:41.708 Info: Docs: line 3: Selected 10040 fragments to
filter.

>From the last line, you can see that index resolution found 10,040
candidate fragments for evaluating the program.  But because it was able
to use a "range ordering constrain" (meaning it used the range index) it
was able to help the server avoid fetching all the fragments off disk.  

Why did we see more than one fragment in the query meters then?  That's
because the server is doing some pre-fetching.

Cheers,

Ron

    Ron Avnur

    Lead Engineer
    Mark Logic Corporation 
    2000 Alameda de las Pulgas 
    Suite 100 
    San Mateo, California  94403 
    +1 650 655 2319 Phone 
    +1 650 655 2310 Fax 
    ron.avnur at marklogic.com 
    www.marklogic.com 

This e-mail and any accompanying attachments are confidential. The
information is intended solely for the use of the individual to whom it
is addressed. Any review, disclosure, copying, distribution, or use of
this e-mail communication by others is strictly prohibited. If you are
not the intended recipient, please notify us immediately by returning
this message to the sender and delete all copies.  Thank you for your
cooperation.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://xqzone.marklogic.com/pipermail/general/attachments/20070227/2353447c/attachment-0001.html