[MarkLogic Dev General] Struggling with Query Time Out

Geert Josten geert.josten at dayon.nl
Fri Mar 16 14:44:50 PDT 2012


Hi Betty,

That positional predicate might cause trouble? How about dropping it and
adding a where to the FLWOR that checks whether
preceding-sibling::ns1:GeneralEvent[1] exists..

Kind regards,
Geert

> -----Oorspronkelijk bericht-----
> Van: general-bounces at developer.marklogic.com [mailto:general-
> bounces at developer.marklogic.com] Namens Betty Harvey
> Verzonden: vrijdag 16 maart 2012 23:04
> Aan: MarkLogic Developer Discussion
> Onderwerp: Re: [MarkLogic Dev General] Struggling with Query Time Out
>
> Thanks!!!
>
> I set an element range index on the main database and have apparently
run
> out of disk space - I will deal with that issue later. It is running on
a
> VM machine.
>
> I also set a range index on EventDate in the 'documents' database for
test
> purposes.   I rewrote the query to use cts:search and it comes back on
the
> 'documents' database that the "Expression is unsearchable" so it looks
> like
> I am not sure what this error message means but I think it might not be
> recognizing the range index.
>
> Am I missing something significant.   The documents have 3 namespaces.
> The EventDate is in the 'ns1' namespace.  I only used one
> cts:element-range-query as a test.
>
> Revised test code:
>
> for $ACE in
>
cts:search(collection()/descendant::ns1:ACE/ns1:EventSet/ns1:GeneralEvent[
1],
>      cts:element-range-query (xs:QName('EventDate'), '<',
> xs:dateTime('2011-03-01T00:00:00') ) )
> let $ACEId := $ACE/ancestor::ns1:ACE/ns1:ACEId
> let $EventDate := $ACE/ns1:EventDate
> return
> <a>
> {$ACEId}
> {$EventDate}
> <time>{xdmp:elapsed-time()}</time>
> </a>
>
> > Hi Betty,
> >
> > Using a cts:search like David suggests could speed up considerably,
> > indeed. You can use xdmp:directory as searchable expression, I
thought,
> > but you can also add it to the query part using cts:directory-query.
> >
> > Note though that if you rewrite the date predicates to
> > cts:element-range-query's, that it may make a lot of difference
whether
> > ACE is a fragment root or not. If you include /descendant::ACE in your
> > searchable path, then the end result is filtered to make sure each ACE
> > matches the query, but there could be a lot of false positives (and
hence
> > xdmp:estimate could return a too high value).
> >
> > Kind regards,
> > Geert
> >
> >> -----Oorspronkelijk bericht-----
> >> Van: general-bounces at developer.marklogic.com [mailto:general-
> >> bounces at developer.marklogic.com] Namens David Lee
> >> Verzonden: vrijdag 16 maart 2012 19:54
> >> Aan: MarkLogic Developer Discussion
> >> Onderwerp: Re: [MarkLogic Dev General] Struggling with Query Time Out
> >>
> >> First off cts:search is exactly what you want for this.
> >> Second you are doing string compares against datetime values.  To
help
> > with this
> >> you may need to create a range index  on EventDate and compare
against
> >> xs:dateTime('xxxxxx')
> >> Thirdly your doing a directory search which you might not actually
need
> > if these
> >> documents are in know namespaces.
> >> But hold off on that until you get the first two worked out.
> >> cts:search() is really your friend in this case, but you do want to
make
> > a range
> >> index so that the system knows the values are dates otherwise "gt"
will
> > do string
> >> not date comparisons
> >>
> >> Once you get both those working your searches should be nearly
instant.
> >>
> >>
> >>
> >
--------------------------------------------------------------------------
> > ---
> >> David Lee
> >> Lead Engineer
> >> MarkLogic Corporation
> >> dlee at marklogic.com
> >> Phone: +1 650-287-2531
> >> Cell:  +1 812-630-7622
> >> www.marklogic.com
> >>
> >> This e-mail and any accompanying attachments are confidential. The
> >> information is intended solely for the use of the individual to whom
it
> > is
> >> addressed. Any review, disclosure, copying, distribution, or use of
this
> > e-mail
> >> communication by others is strictly prohibited. If you are not the
> > intended
> >> recipient, please notify us immediately by returning this message to
the
> > sender
> >> and delete all copies. Thank you for your cooperation.
> >>
> >>
> >> > -----Original Message-----
> >> > From: general-bounces at developer.marklogic.com [mailto:general-
> >> > bounces at developer.marklogic.com] On Behalf Of Betty Harvey
> >> > Sent: Friday, March 16, 2012 3:17 PM
> >> > To: MarkLogic Developer Discussion
> >> > Subject: [MarkLogic Dev General] Struggling with Query Time Out
> >> >
> >> > I have been unable to get this query to run successfully without
> > timing
> >> > out.  To make sure my logic was correct I placed 100 documents in
the
> >> > 'documents' database and query runs successfully and very quickly.
In
> > the
> >> > large database 1.7 million objects the query always times out.
> >> >
> >> > I am not sure cts:search will help.  I played around with it
without
> >> > success.   The goal of the query is to gather information for a
> > particular
> >> > month based on when the document was created.   Below is the code:
> >> >
> >> >
> >> > for $ACE in xdmp:directory('opt/MOR/ACE/')/descendant::ACE
> >> >    [EventSet/GeneralEvent[1]/EventDate gt '2011-03-01T00:00:00']
> >> >    [EventSet/era:GeneralEvent[1]/EventDate lt
'2011-04-01T00:00:00']
> >> > let $ACEId := $ACE/ACEId
> >> > let $EventDate := $ACE/EventSet/era:GeneralEvent[1]/era:EventDate
> >> > return
> >> > <a>
> >> > {$ACEId}
> >> > {$EventDate}
> >> > </a>
> >> >
> >> > Any ideas are appreciated!
> >> >
> >> > Betty
> >> >
> >> > _______________________________________________
> >> > General mailing list
> >> > General at developer.marklogic.com
> >> > http://developer.marklogic.com/mailman/listinfo/general
> >> _______________________________________________
> >> General mailing list
> >> General at developer.marklogic.com
> >> http://developer.marklogic.com/mailman/listinfo/general
> > _______________________________________________
> > General mailing list
> > General at developer.marklogic.com
> > http://developer.marklogic.com/mailman/listinfo/general
> >
>
>
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
> Betty Harvey                         | Phone:  410-787-9200  FAX: 9830
> Electronic Commerce Connection, Inc. |
> harvey at eccnet.com                    | Washington,DC XML Users Grp
> URL:  http://www.eccnet.com          | http://www.eccnet.com/xmlug
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\\/\/
> Member of XML Guild (www.xmlguild.org)
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general


More information about the General mailing list