[MarkLogic Dev General] Struggling with Query Time Out

Evan Lenz Evan.Lenz at marklogic.com
Tue Mar 20 06:25:49 PDT 2012


As a quick tip, Betty, you can easily check whether a given expression is searchable or not by using Query Console. I just ran this:

declare namespace ns1="whatever";
xdmp:plan(
collection()/descendant::ns1:ACE/ns1:EventSet/ns1:GeneralEvent[1]
)

Whose output included this:
  <qry:info-trace>Step 4 is unsearchable: ns1:GeneralEvent[1]</qry:info-trace>

This tells me where the problem is, and verifies Mike's suspicion.

Evan


On 3/19/12 4:16 PM, "Michael Blakeley" <mike at blakeley.com<mailto:mike at blakeley.com>> wrote:

Betty, I think it's the '[1]' that makes that expression unsearchable. Normally the XPath indexes simply record the presence of elements, not their position.

-- Mike

On 16 Mar 2012, at 15:03 , Betty Harvey wrote:

Thanks!!!
I set an element range index on the main database and have apparently run
out of disk space - I will deal with that issue later. It is running on a
VM machine.
I also set a range index on EventDate in the 'documents' database for test
purposes.   I rewrote the query to use cts:search and it comes back on the
'documents' database that the "Expression is unsearchable" so it looks
like
I am not sure what this error message means but I think it might not be
recognizing the range index.
Am I missing something significant.   The documents have 3 namespaces.
The EventDate is in the 'ns1' namespace.  I only used one
cts:element-range-query as a test.
Revised test code:
for $ACE in
cts:search(collection()/descendant::ns1:ACE/ns1:EventSet/ns1:GeneralEvent[1],
     cts:element-range-query (xs:QName('EventDate'), '<',
xs:dateTime('2011-03-01T00:00:00') ) )
let $ACEId := $ACE/ancestor::ns1:ACE/ns1:ACEId
let $EventDate := $ACE/ns1:EventDate
return
<a>
{$ACEId}
{$EventDate}
<time>{xdmp:elapsed-time()}</time>
</a>
Hi Betty,
Using a cts:search like David suggests could speed up considerably,
indeed. You can use xdmp:directory as searchable expression, I thought,
but you can also add it to the query part using cts:directory-query.
Note though that if you rewrite the date predicates to
cts:element-range-query's, that it may make a lot of difference whether
ACE is a fragment root or not. If you include /descendant::ACE in your
searchable path, then the end result is filtered to make sure each ACE
matches the query, but there could be a lot of false positives (and hence
xdmp:estimate could return a too high value).
Kind regards,
Geert
-----Oorspronkelijk bericht-----
Van: general-bounces at developer.marklogic.com<mailto:general-bounces at developer.marklogic.com> [mailto:general-
bounces at developer.marklogic.com<mailto:bounces at developer.marklogic.com>] Namens David Lee
Verzonden: vrijdag 16 maart 2012 19:54
Aan: MarkLogic Developer Discussion
Onderwerp: Re: [MarkLogic Dev General] Struggling with Query Time Out
First off cts:search is exactly what you want for this.
Second you are doing string compares against datetime values.  To help
with this
you may need to create a range index  on EventDate and compare against
xs:dateTime('xxxxxx')
Thirdly your doing a directory search which you might not actually need
if these
documents are in know namespaces.
But hold off on that until you get the first two worked out.
cts:search() is really your friend in this case, but you do want to make
a range
index so that the system knows the values are dates otherwise "gt" will
do string
not date comparisons
Once you get both those working your searches should be nearly instant.
--------------------------------------------------------------------------
---
David Lee
Lead Engineer
MarkLogic Corporation
dlee at marklogic.com<mailto:dlee at marklogic.com>
Phone: +1 650-287-2531
Cell:  +1 812-630-7622
www.marklogic.com
This e-mail and any accompanying attachments are confidential. The
information is intended solely for the use of the individual to whom it
is
addressed. Any review, disclosure, copying, distribution, or use of this
e-mail
communication by others is strictly prohibited. If you are not the
intended
recipient, please notify us immediately by returning this message to the
sender
and delete all copies. Thank you for your cooperation.
-----Original Message-----
From: general-bounces at developer.marklogic.com<mailto:general-bounces at developer.marklogic.com> [mailto:general-
bounces at developer.marklogic.com<mailto:bounces at developer.marklogic.com>] On Behalf Of Betty Harvey
Sent: Friday, March 16, 2012 3:17 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Struggling with Query Time Out
I have been unable to get this query to run successfully without
timing
out.  To make sure my logic was correct I placed 100 documents in the
'documents' database and query runs successfully and very quickly. In
the
large database 1.7 million objects the query always times out.
I am not sure cts:search will help.  I played around with it without
success.   The goal of the query is to gather information for a
particular
month based on when the document was created.   Below is the code:
for $ACE in xdmp:directory('opt/MOR/ACE/')/descendant::ACE
   [EventSet/GeneralEvent[1]/EventDate gt '2011-03-01T00:00:00']
   [EventSet/era:GeneralEvent[1]/EventDate lt '2011-04-01T00:00:00']
let $ACEId := $ACE/ACEId
let $EventDate := $ACE/EventSet/era:GeneralEvent[1]/era:EventDate
return
<a>
{$ACEId}
{$EventDate}
</a>
Any ideas are appreciated!
Betty
_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Betty Harvey                         | Phone:  410-787-9200  FAX: 9830
Electronic Commerce Connection, Inc. |
harvey at eccnet.com<mailto:harvey at eccnet.com>                    | Washington,DC XML Users Grp
URL:  http://www.eccnet.com          | http://www.eccnet.com/xmlug
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\\/\/
Member of XML Guild (www.xmlguild.org)
_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general

_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://developer.marklogic.com/pipermail/general/attachments/20120320/46f5f7dc/attachment-0001.html 


More information about the General mailing list