[MarkLogic Dev General] Struggling with Query Time Out

Michael Blakeley mike at blakeley.com
Mon Mar 19 13:16:07 PDT 2012


Betty, I think it's the '[1]' that makes that expression unsearchable. Normally the XPath indexes simply record the presence of elements, not their position.

-- Mike

On 16 Mar 2012, at 15:03 , Betty Harvey wrote:

> Thanks!!!
> 
> I set an element range index on the main database and have apparently run 
> out of disk space - I will deal with that issue later. It is running on a
> VM machine.
> 
> I also set a range index on EventDate in the 'documents' database for test
> purposes.   I rewrote the query to use cts:search and it comes back on the
> 'documents' database that the "Expression is unsearchable" so it looks
> like
> I am not sure what this error message means but I think it might not be
> recognizing the range index.
> 
> Am I missing something significant.   The documents have 3 namespaces. 
> The EventDate is in the 'ns1' namespace.  I only used one
> cts:element-range-query as a test.
> 
> Revised test code:
> 
> for $ACE in
> cts:search(collection()/descendant::ns1:ACE/ns1:EventSet/ns1:GeneralEvent[1],
>     cts:element-range-query (xs:QName('EventDate'), '<',
> xs:dateTime('2011-03-01T00:00:00') ) )
> let $ACEId := $ACE/ancestor::ns1:ACE/ns1:ACEId
> let $EventDate := $ACE/ns1:EventDate
> return
> <a>
> {$ACEId}
> {$EventDate}
> <time>{xdmp:elapsed-time()}</time>
> </a>
> 
>> Hi Betty,
>> 
>> Using a cts:search like David suggests could speed up considerably,
>> indeed. You can use xdmp:directory as searchable expression, I thought,
>> but you can also add it to the query part using cts:directory-query.
>> 
>> Note though that if you rewrite the date predicates to
>> cts:element-range-query's, that it may make a lot of difference whether
>> ACE is a fragment root or not. If you include /descendant::ACE in your
>> searchable path, then the end result is filtered to make sure each ACE
>> matches the query, but there could be a lot of false positives (and hence
>> xdmp:estimate could return a too high value).
>> 
>> Kind regards,
>> Geert
>> 
>>> -----Oorspronkelijk bericht-----
>>> Van: general-bounces at developer.marklogic.com [mailto:general-
>>> bounces at developer.marklogic.com] Namens David Lee
>>> Verzonden: vrijdag 16 maart 2012 19:54
>>> Aan: MarkLogic Developer Discussion
>>> Onderwerp: Re: [MarkLogic Dev General] Struggling with Query Time Out
>>> 
>>> First off cts:search is exactly what you want for this.
>>> Second you are doing string compares against datetime values.  To help
>> with this
>>> you may need to create a range index  on EventDate and compare against
>>> xs:dateTime('xxxxxx')
>>> Thirdly your doing a directory search which you might not actually need
>> if these
>>> documents are in know namespaces.
>>> But hold off on that until you get the first two worked out.
>>> cts:search() is really your friend in this case, but you do want to make
>> a range
>>> index so that the system knows the values are dates otherwise "gt" will
>> do string
>>> not date comparisons
>>> 
>>> Once you get both those working your searches should be nearly instant.
>>> 
>>> 
>>> 
>> --------------------------------------------------------------------------
>> ---
>>> David Lee
>>> Lead Engineer
>>> MarkLogic Corporation
>>> dlee at marklogic.com
>>> Phone: +1 650-287-2531
>>> Cell:  +1 812-630-7622
>>> www.marklogic.com
>>> 
>>> This e-mail and any accompanying attachments are confidential. The
>>> information is intended solely for the use of the individual to whom it
>> is
>>> addressed. Any review, disclosure, copying, distribution, or use of this
>> e-mail
>>> communication by others is strictly prohibited. If you are not the
>> intended
>>> recipient, please notify us immediately by returning this message to the
>> sender
>>> and delete all copies. Thank you for your cooperation.
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: general-bounces at developer.marklogic.com [mailto:general-
>>>> bounces at developer.marklogic.com] On Behalf Of Betty Harvey
>>>> Sent: Friday, March 16, 2012 3:17 PM
>>>> To: MarkLogic Developer Discussion
>>>> Subject: [MarkLogic Dev General] Struggling with Query Time Out
>>>> 
>>>> I have been unable to get this query to run successfully without
>> timing
>>>> out.  To make sure my logic was correct I placed 100 documents in the
>>>> 'documents' database and query runs successfully and very quickly. In
>> the
>>>> large database 1.7 million objects the query always times out.
>>>> 
>>>> I am not sure cts:search will help.  I played around with it without
>>>> success.   The goal of the query is to gather information for a
>> particular
>>>> month based on when the document was created.   Below is the code:
>>>> 
>>>> 
>>>> for $ACE in xdmp:directory('opt/MOR/ACE/')/descendant::ACE
>>>>   [EventSet/GeneralEvent[1]/EventDate gt '2011-03-01T00:00:00']
>>>>   [EventSet/era:GeneralEvent[1]/EventDate lt '2011-04-01T00:00:00']
>>>> let $ACEId := $ACE/ACEId
>>>> let $EventDate := $ACE/EventSet/era:GeneralEvent[1]/era:EventDate
>>>> return
>>>> <a>
>>>> {$ACEId}
>>>> {$EventDate}
>>>> </a>
>>>> 
>>>> Any ideas are appreciated!
>>>> 
>>>> Betty
>>>> 
>>>> _______________________________________________
>>>> General mailing list
>>>> General at developer.marklogic.com
>>>> http://developer.marklogic.com/mailman/listinfo/general
>>> _______________________________________________
>>> General mailing list
>>> General at developer.marklogic.com
>>> http://developer.marklogic.com/mailman/listinfo/general
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> http://developer.marklogic.com/mailman/listinfo/general
>> 
> 
> 
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
> Betty Harvey                         | Phone:  410-787-9200  FAX: 9830
> Electronic Commerce Connection, Inc. |
> harvey at eccnet.com                    | Washington,DC XML Users Grp
> URL:  http://www.eccnet.com          | http://www.eccnet.com/xmlug
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\\/\/
> Member of XML Guild (www.xmlguild.org)
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
> 



More information about the General mailing list