[MarkLogic Dev General] XDMP-EXPNTREECACHEFULL error

Gary Larsen gary.larsen at envisn.com
Thu Jul 2 04:23:59 PDT 2015


Hi Indy,

 

I've been using this syntax for cts:search queries with large result sets to
avoid XDMP-EXPNTREECACHEFULL.

 

 

(: setup constraining query :)

let $cq := 
    ...
    )) 

        let $incr := 1000
        let $size := xdmp:estimate(cts:search(doc(), $cq, 'unfiltered')) 
        let $segs := ceiling($size div $incr) return 
        
        for $x in (1 to $segs) 
            let $start :=  (($x -1) * $incr) +1 
            let $end := $start + $incr -1 return
            for $a in  cts:search(doc(), $cq, 'unfiltered')[$start to
$end]/content  return

 

              ... 



 

Gary 

 

From: general-bounces at developer.marklogic.com
[mailto:general-bounces at developer.marklogic.com] On Behalf Of ???
Sent: Thursday, July 02, 2015 4:54 AM
To: general at developer.marklogic.com
Subject: Re: [MarkLogic Dev General] XDMP-EXPNTREECACHEFULL error

 

I had created range index on delete_flag and report_date, and i change my
xquery to this:
count(cts:search(collection('test')/patient/link/visit/lab_report[@delete_fl
ag = '0'],
  cts:and-query(
 
(cts:element-attribute-range-query(xs:QName("lab_report"),xs:QName("report_d
ate"), ">=", xs:dateTime("2010-09-25T00:00:00.000")),
 
cts:element-attribute-range-query(xs:QName("lab_report"),xs:QName("report_da
te"), "<=", xs:dateTime("2014-09-25T00:00:00.000"))))));
 
and this query can work.
 
but if i execute the follow xquery, MarkLogic still will throw an
XDMP-EXPNTREECACHEFULL error:
cts:search(collection()/patient/link/visit/lab_report[@delete_flag = '0'],
  cts:and-query(
 
(cts:element-attribute-range-query(xs:QName("lab_report"),xs:QName("report_d
ate"), ">=", xs:dateTime("2010-09-25T00:00:00.000")),
 
cts:element-attribute-range-query(xs:QName("lab_report"),xs:QName("report_da
te"), "<=", xs:dateTime("2014-09-25T00:00:00.000")))));
 
the above xquery will return a very large resut set, in this case, what can
i do to avoid the  XDMP-EXPNTREECACHEFULL error
 
 
> 
>Message: 1
>Date: Wed, 1 Jul 2015 10:58:24 +0530
>From: Indrajeet Verma < <mailto:indrajeet.verma at gmail.com>
indrajeet.verma at gmail.com>
>Subject: Re: [MarkLogic Dev General] XDMP-EXPNTREECACHEFULL error
>To: MarkLogic Developer Discussion <general at developer.marklogic.com>
>Message-ID:
>       <CAKwHAegeH70RO8wgqvMuv3JAqbCmV7U09gnXHNfXmFbV1OrvpA at mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
> 
>In Addition of Asit response,
> 
>You should go with cts:element-values() and you should create cts:and-query
>and inside it cts:range-query for the dates and flag.
> 
>Create range index for lab_report, report_date and delete_flag.
> 
>Let me know if you want any help in creating query.
> 
>Regards,
>Indy
> 
>On Wed, Jul 1, 2015 at 10:45 AM, Asit Nautiyal <asitnautiyal at outlook.com>
>wrote:
> 
>> Hi Zise,
>> 
>> By observing below code, it will hit performance for sure and that's why
>> your are getting XDMP-EXPNTREECACHEFULL error.
>> To avoid XDMP-EXPNTREECACHEFULL error you have to tune your code. Use
>> cts:queries (like range queries and others) to get your result first and
>> then you can use count.
>> 
>> Please go through the below link :
>> 
>> https://docs.marklogic.com/guide/performance
>> 
>> 
>> Regards,
>> Asit Nautiyal
>> ------------------------------
>> Date: Wed, 1 Jul 2015 10:11:01 +0800
>> From: zisedeqing at 163.com
>> To: general at developer.marklogic.com
>> Subject: [MarkLogic Dev General] XDMP-EXPNTREECACHEFULL error
>> 
>> 
>> when i execute the following xquery , marklogic throw an
>> XDMP-EXPNTREECACHEFULL error:
>> count(for $i in collection('test')/patient/link/visit/lab_report
>> where $i /@report_date >= '2010-09-25T00:00:00.000' and $i/@ report_date
>> <= '2014-09-25T00:00:00.000' and $i/@delete_flag = '0'
>> return $i)
>> 
>> I can not increase size of expanded tree cache, so what i can do to avoid
>> the XDMP-EXPNTREECACHEFULL error while the query resturn the very large
>> results
>> 
>> 
>> 
>> _______________________________________________ General mailing list
>> General at developer.marklogic.com Manage your subscription at:
>> http://developer.marklogic.com/mailman/listinfo/general
>> 
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> Manage your subscription at:
>> http://developer.marklogic.com/mailman/listinfo/general
>> 
>> 
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL:
http://developer.marklogic.com/pipermail/general/attachments/20150701/e5ad4c
84/attachment-0001.html 
> 
>------------------------------
> 
>Message: 2
>Date: Wed, 1 Jul 2015 13:46:50 +0000
>From: Dave Cassel <Dave.Cassel at marklogic.com>
>Subject: Re: [MarkLogic Dev General] how to update the attribute of
>       nodes?
>To: MarkLogic Developer Discussion <general at developer.marklogic.com>
>Message-ID: <D1B93D2E.CCA51%dave.cassel at marklogic.com>
>Content-Type: text/plain; charset="us-ascii"
> 
>To Asit's point, if you need to make an update like this across a large
number of documents, take a look at
Corb2<https://github.com/marklogic/corb2>.
> 
>--
>Dave Cassel<http://davidcassel.net>,
@dmcassel<https://twitter.com/dmcassel>
>Technical Community Manager
>MarkLogic Corporation<http://www.marklogic.com/>
>http://developer.marklogic.com/
> 
> 
>From: Indrajeet Verma <
<mailto:indrajeet.verma at gmail.com%3cmailto:indrajeet.verma at gmail.com>
indrajeet.verma at gmail.com<mailto:indrajeet.verma at gmail.com>>
>Reply-To: MarkLogic Developer Discussion <general at developer.marklogic.com
<mailto:general at developer.marklogic.com%3cmailto:general at developer.marklogic
.com> <mailto:general at developer.marklogic.com>>
>Date: Wednesday, July 1, 2015 at 1:21 AM
>To: MarkLogic Developer Discussion <general at developer.marklogic.com
<mailto:general at developer.marklogic.com%3cmailto:general at developer.marklogic
.com> <mailto:general at developer.marklogic.com>>
>Subject: Re: [MarkLogic Dev General] how to update the attribute of nodes?
> 
>Hi Zise,
> 
>You can try below code as well,
> 
>xdmp:node-replace(/catalog/product/name/@language, attribute language
{"chinese"})
> 
>Regards,
>Indy
> 
>On Wed, Jul 1, 2015 at 10:37 AM, Asit Nautiyal <asitnautiyal at outlook.com
<mailto:asitnautiyal at outlook.com%3cmailto:asitnautiyal at outlook.com>
<mailto:asitnautiyal at outlook.com>> wrote:
>Hi Zise,
> 
>There are various ways to do this simple task. Find the below sample code :
> 
>let $doc := fn:doc("/xmlFile.xml")
>let $oldNode := $doc/catalog/product/name[@language="en"]
>let $newNode := element {fn:local-name($oldNode)} { attribute language
{"chinese"}, $oldNode/text()}
> 
>return xdmp:node-replace($oldNode,$newNode)
> 
>Above code will work as expected. But, if your database size is very huge
and you want to perform such update on entire database then use cts:queries
for better performance.
> 
> 
>Regards,
>Asit Nautiyal
> 
>________________________________
>Date: Wed, 1 Jul 2015 11:50:02 +0800
>From: zisedeqing at 163.com
<mailto:zisedeqing at 163.com%3cmailto:zisedeqing at 163.com>
<mailto:zisedeqing at 163.com>
>To: general at developer.marklogic.com
<mailto:general at developer.marklogic.com%3cmailto:general at developer.marklogic
.com> <mailto:general at developer.marklogic.com>
>Subject: [MarkLogic Dev General] how to update the attribute of nodes?
> 
> 
>the xml document is:
><catalog>
><productdept="ACC">
><number>
>443
></number>
><namelanguage="en">
>Deluxe Travel Bag
></name>
></product>
></catalog>
> 
>I want to update the attirbute language of node name to new value, after
update operation, the name node is:
><namelanguage="chinese">
>Deluxe Travel Bag
></name>
> 
>which function can do this in marklogic 8.0?
> 
> 
> 
>_______________________________________________ General mailing list
General at developer.marklogic.com
<mailto:General at developer.marklogic.com%3cmailto:General at developer.marklogic
.com> <mailto:General at developer.marklogic.com> Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
> 
>_______________________________________________
>General mailing list
>General at developer.marklogic.com
<mailto:General at developer.marklogic.com%3cmailto:General at developer.marklogic
.com> <mailto:General at developer.marklogic.com>
>Manage your subscription at:
>http://developer.marklogic.com/mailman/listinfo/general
> 
> 
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL:
http://developer.marklogic.com/pipermail/general/attachments/20150701/0322e4
50/attachment.html 
> 
>------------------------------
> 
>_______________________________________________
>General mailing list
>General at developer.marklogic.com
>Manage your subscription at: 
>http://developer.marklogic.com/mailman/listinfo/general
> 
> 
>End of General Digest, Vol 133, Issue 1
>***************************************

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://developer.marklogic.com/pipermail/general/attachments/20150702/1f09ca20/attachment-0001.html 


More information about the General mailing list