Thanks Geert,<div><br></div><div>Is there any alternate solution to find the unique elements within a database?</div><div><br></div><div>Warm Regards,</div><div>Vishnu<br><div><br></div><div><br><br><div class="gmail_quote">
On Mon, Mar 26, 2012 at 5:55 PM, Geert Josten <span dir="ltr">&lt;<a href="mailto:geert.josten@dayon.nl">geert.josten@dayon.nl</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="NL" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Hi Vishnu,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">90 mb isn’t much indeed, but MarkLogic is configured to keep a low memory footprint, even if there are 30 concurrent requests. To make that sure, the tree size limit (look at the database setting in the admin interface) is usually pretty low. I have 8Gb and still it is set to no more than 85mb by default. But you can increase it if you like.</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">A more streaming approach like my advice attempts to achieve to some extend helps keeping the footprint low, and keep MarkLogic fast.</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Kind regards,</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Geert</span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p>

<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">Van:</span></b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> <a href="mailto:general-bounces@developer.marklogic.com" target="_blank">general-bounces@developer.marklogic.com</a> [mailto:<a href="mailto:general-bounces@developer.marklogic.com" target="_blank">general-bounces@developer.marklogic.com</a>] <b>Namens </b>VISH RAJPUT<br>

<b>Verzonden:</b> maandag 26 maart 2012 14:17<br><b>Aan:</b> MarkLogic Developer Discussion<br><b>Onderwerp:</b> Re: [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL</span></p></div></div><div><div class="h5">
<p class="MsoNormal">
 </p><p class="MsoNormal">Thanks Geert,</p><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">But still it shows <b><span style="font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;background:white">XDMP-EXPNTREECACHEFULL: distinct-values(collection(&quot;ContentAnalysis&quot;)//*/local-name()) -- Expanded tree cache full on host.... </span></b>the database overall size is only 90MB i don&#39;t think it is so huge data for marklogic....</p>

</div><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">Regards,</p></div><div><p class="MsoNormal">Vishnu</p></div><div><div><p class="MsoNormal" style="margin-bottom:12.0pt">

 </p><div><p class="MsoNormal">On Mon, Mar 26, 2012 at 1:25 PM, Geert Josten &lt;<a href="mailto:geert.josten@dayon.nl" target="_blank">geert.josten@dayon.nl</a>&gt; wrote:</p><div><div><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Hi Vishnu,</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Your FLWOR expression won’t return distinct names, since you are applying the function to each individual name. You should write:</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal" style="text-indent:35.4pt"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">distinct-values(</span></p>

<p class="MsoNormal" style="text-indent:35.4pt"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">    for $a in //*</span></p><p class="MsoNormal" style="text-indent:35.4pt">

<span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">    return $a</span></p><p class="MsoNormal" style="text-indent:35.4pt"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">)</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Or better:</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal" style="text-indent:35.4pt"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">distinct-values(collection()//*/local-name())</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">But this still might not perform well, or still max out on list or tree caches. This approach is creating a complete list of all element names first, and starts applying distinct-values only thereafter. You might consider taking multiple steps, like per doc first, and then clustering per 100 files, and only then all clusters. You could also just take 100 random samples, and use that. That doesn’t guarantee a 100% complete list, but it remains performant even if your database grows 10 or 100 fold.</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Kind regards,</span></p>

<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Geert</span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"> </span></p>

<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">Van:</span></b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> <a href="mailto:general-bounces@developer.marklogic.com" target="_blank">general-bounces@developer.marklogic.com</a> [mailto:<a href="mailto:general-bounces@developer.marklogic.com" target="_blank">general-bounces@developer.marklogic.com</a>] <b>Namens </b>VISH RAJPUT<br>

<b>Verzonden:</b> maandag 26 maart 2012 8:29<br><b>Aan:</b> <a href="mailto:general@developer.marklogic.com" target="_blank">general@developer.marklogic.com</a><br><b>Onderwerp:</b> [MarkLogic Dev General] Fwd: [1.0-ml] XDMP-EXPNTREECACHEFULL</span></p>

</div></div><div><div><p class="MsoNormal"> </p><p class="MsoNormal" style="margin-bottom:12.0pt">The size of the all files is 90 MB approx.</p><div><p class="MsoNormal">---------- Forwarded message ----------<br>
From: <b>VISH RAJPUT</b> &lt;<a href="mailto:svishnu.singh4@gmail.com" target="_blank">svishnu.singh4@gmail.com</a>&gt;<br>Date: Mon, Mar 26, 2012 at 11:56 AM<br>Subject: [1.0-ml] XDMP-EXPNTREECACHEFULL<br>To: <a href="mailto:general@developer.marklogic.com" target="_blank">general@developer.marklogic.com</a><br>

<br><br>Hi,</p><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">I have 2000 files in Marklogic database within a single forest and i want to find out the unique element name from this database for the whole 2000 files. For this i wrote the below query:-</p>

</div><div><p class="MsoNormal"> </p></div><div><div><p class="MsoNormal">for $a in //*</p></div><div><p class="MsoNormal">return distinct-values($a/local-name()))</p></div></div><div><p class="MsoNormal">
 </p></div><div><p class="MsoNormal">but by this i got an error &quot;<b><span style="font-family:&quot;Arial&quot;,&quot;sans-serif&quot;">[1.0-ml] XDMP-EXPNTREECACHEFULL&quot; </span></b> what should i do?</p></div>
<div><p class="MsoNormal"> </p></div><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">Regards,</p></div><div><p class="MsoNormal">Vishnu Singh</p></div></div><p class="MsoNormal">
 </p></div></div></div></div></div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>_______________________________________________<br>General mailing list<br><a href="mailto:General@developer.marklogic.com" target="_blank">General@developer.marklogic.com</a><br>

<a href="http://developer.marklogic.com/mailman/listinfo/general" target="_blank">http://developer.marklogic.com/mailman/listinfo/general</a></p></div><p class="MsoNormal"> </p></div></div></div></div></div></div></div>
<br>_______________________________________________<br>
General mailing list<br>
<a href="mailto:General@developer.marklogic.com">General@developer.marklogic.com</a><br>
<a href="http://developer.marklogic.com/mailman/listinfo/general" target="_blank">http://developer.marklogic.com/mailman/listinfo/general</a><br>
<br></blockquote></div><br></div></div>