[MarkLogic Dev General] SVC:EXTIME -- Time limit exceeded

Will Thompson wthompson at jonesmcclure.com
Tue Mar 13 08:32:44 PDT 2012


Jaspreet,

You are looking for cache misses, so this doesn't indicate any index settings in need of adjustment. Value cache misses are expected and aren't related to database settings.

It says here your query is taking 10s, which still seems slow, but it should not be timing out at over 30s.

-W


From: general-bounces at developer.marklogic.com [mailto:general-bounces at developer.marklogic.com] On Behalf Of JASPREET SINGH
Sent: Monday, March 12, 2012 9:13 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] SVC:EXTIME -- Time limit exceeded


Will,

I ran the xdmp:query-meters() on my query and i got the following result:-

<qm:query-meters xsi:schemaLocation="http://marklogic.com/xdmp/query-meters query-meters.xsd" xmlns:qm="http://marklogic.com/xdmp/query-meters" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <qm:elapsed-time>PT10.299693S</qm:elapsed-time>

  <qm:requests>0</qm:requests>

  <qm:list-cache-hits>201</qm:list-cache-hits>

  <qm:list-cache-misses>0</qm:list-cache-misses>

  <qm:in-memory-list-hits>0</qm:in-memory-list-hits>

  <qm:expanded-tree-cache-hits>21</qm:expanded-tree-cache-hits>

  <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

  <qm:compressed-tree-cache-hits>0</qm:compressed-tree-cache-hits>

  <qm:compressed-tree-cache-misses>0</qm:compressed-tree-cache-misses>

  <qm:in-memory-compressed-tree-hits>0</qm:in-memory-compressed-tree-hits>

  <qm:value-cache-hits>5009</qm:value-cache-hits>

  <qm:value-cache-misses>96428</qm:value-cache-misses>

  <qm:regexp-cache-hits>136</qm:regexp-cache-hits>

  <qm:regexp-cache-misses>13</qm:regexp-cache-misses>

  <qm:link-cache-hits>0</qm:link-cache-hits>

  <qm:link-cache-misses>0</qm:link-cache-misses>

  <qm:filter-hits>0</qm:filter-hits>

  <qm:filter-misses>0</qm:filter-misses>

  <qm:fragments-added>0</qm:fragments-added>

  <qm:fragments-deleted>0</qm:fragments-deleted>

  <qm:fs-program-cache-hits>23</qm:fs-program-cache-hits>

  <qm:fs-program-cache-misses>1</qm:fs-program-cache-misses>

  <qm:db-program-cache-hits>0</qm:db-program-cache-hits>

  <qm:db-program-cache-misses>0</qm:db-program-cache-misses>

  <qm:env-program-cache-hits>0</qm:env-program-cache-hits>

  <qm:env-program-cache-misses>0</qm:env-program-cache-misses>

  <qm:fs-main-module-sequence-cache-hits>0</qm:fs-main-module-sequence-cache-hits>

  <qm:fs-main-module-sequence-cache-misses>0</qm:fs-main-module-sequence-cache-misses>

  <qm:db-main-module-sequence-cache-hits>0</qm:db-main-module-sequence-cache-hits>

  <qm:db-main-module-sequence-cache-misses>0</qm:db-main-module-sequence-cache-misses>

  <qm:fs-library-module-cache-hits>20</qm:fs-library-module-cache-hits>

  <qm:fs-library-module-cache-misses>0</qm:fs-library-module-cache-misses>

  <qm:db-library-module-cache-hits>0</qm:db-library-module-cache-hits>

  <qm:db-library-module-cache-misses>0</qm:db-library-module-cache-misses>

  <qm:fragments>

    <qm:fragment>

      <qm:root xmlns="">Publisher</qm:root>

      <qm:expanded-tree-cache-hits>20</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:fragment>

    <qm:fragment>

      <qm:root xmlns="http://www.marklogic.com/ps/lib/lib-search/config">config</qm:root>

      <qm:expanded-tree-cache-hits>1</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:fragment>

  </qm:fragments>

  <qm:documents>

    <qm:document>

      <qm:uri>/resources/config-search.xml</qm:uri>

      <qm:expanded-tree-cache-hits>1</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11536/VOL=2010.5/ISU=4/ART=2010_23/11536_2010_Article_23.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11534/VOL=2010.8/ISU=4/ART=2009_166/11534_2009_Article_166.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11442/VOL=2010.20/ISU=4/ART=2010_581/11442_2010_Article_581.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11756/VOL=2010.65/ISU=4/ART=2010_60/11756_2010_Article_60.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11756/VOL=2010.65/ISU=4/ART=2010_74/11756_2010_Article_74.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11442/VOL=2010.20/ISU=4/ART=2010_483/11442_2010_Article_483.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11534/VOL=2010.8/ISU=4/ART=2009_149/11534_2009_Article_149.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11536/VOL=2010.5/ISU=4/ART=2009_93/11536_2009_Article_93.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11442/VOL=2010.20/ISU=4/ART=2010_599/11442_2010_Article_599.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

    <qm:document>

      <qm:uri>/content/JOU=11536/VOL=2010.5/ISU=4/ART=2009_139/11536_2009_Article_139.xml</qm:uri>

      <qm:expanded-tree-cache-hits>2</qm:expanded-tree-cache-hits>

      <qm:expanded-tree-cache-misses>0</qm:expanded-tree-cache-misses>

    </qm:document>

  </qm:documents>

  <qm:hosts/>

</qm:query-meters>

Is there anything informative to optimize the query through database settings??

Jaspreet

On Mon, Mar 12, 2012 at 11:49 PM, Will Thompson <wthompson at jonesmcclure.com<mailto:wthompson at jonesmcclure.com>> wrote:
Jaspreet,

"Unsearchable" according to xdmp:plan just means that it is not an inline fully searchable path expression. MarkLogic elaborates on this in the docs. It doesn't mean that your cts:element-values() call isn't working correctly, just that it's not a valid expression for cts:search.

Regarding performance, you could try xdmp:query-meters() as Jon lays out in his blog post and see what is happening under the hood of your database.

-Will


From: general-bounces at developer.marklogic.com<mailto:general-bounces at developer.marklogic.com> [mailto:general-bounces at developer.marklogic.com<mailto:general-bounces at developer.marklogic.com>] On Behalf Of JASPREET SINGH
Sent: Monday, March 12, 2012 4:09 AM

To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] SVC:EXTIME -- Time limit exceeded


Hi Damon,

I ran the profiler on cq for the following query which was taking time and I passed the hard-coded values where required.

let $options := ("document","collation=http://marklogic.com/collation/","frequency-order")










let $base-query := cts:directory-query("/content/", "infinity")
let $values := cts:element-values(fn:QName("http://www.xyz.com/app/meta","Country"), "", $options, $base-query)










return fn:count($values)


According to profiler,

cts:element-values(fn:QName("http://www.xyz.com/app/meta","Country"), "", $options, $base-query) has the maximum shallow-%.










Then I ran xdmp:plan() on the same statement :-

cts:element-values(fn:QName("http://www.xyz.com/app/meta","Country"), "", $options, $base-query)










and I got the following result :-

<qry:query-plan xmlns:qry="http://marklogic.com/cts/query">

  <qry:error-trace><error:error xmlns:error="http://marklogic.com/xdmp/error" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

      <error:code>XDMP-UNSEARCHABLE</error:code>

      <error:name/>

      <error:xquery-version>0.9-ml</error:xquery-version>

      <error:message>Expression is unsearchable</error:message>

      <error:format-string>XDMP-UNSEARCHABLE: xdmp:plan(cts:element-values(fn:expanded-QName("http://www.xyz.com/app/meta", "Country"), "", ("document", "collation=http://marklogic.com/collation/", "frequency-order"), cts:directory-query("/content/", "infinity"))) -- Expression is unsearchable</error:format-string>

      <error:retryable>false</error:retryable>

      <error:expr>xdmp:plan(cts:element-values(fn:expanded-QName("http://www.xyz.com/app/meta", "Country"), "", ("document", "collation=http://marklogic.com/collation/", "frequency-order"), cts:directory-query("/content/", "infinity")))</error:expr>

      <error:data/>

      <error:stack>

               <error:frame>

                 <error:line>4</error:line>

                 <error:operation>xdmp:eval("let $options := (&amp;quot;document&amp;quot;,&amp;quot;collation=http://mar.<http://mar./>..", (), &lt;options xmlns="xdmp:eval"&gt;&lt;database&gt;12438542817496803555&lt;/database&gt;&lt;default-xquery-version...&lt;/options&gt;)</error:operation>

                 <error:variables>

                   <error:variable>

                     <error:name xmlns="">options</error:name>

                     <error:value>("document", "collation=http://marklogic.com/collation/", "frequency-order")</error:value>

                   </error:variable>

                   <error:variable>

                     <error:name xmlns="">base-query</error:name>

                     <error:value>cts:directory-query("/content/", "infinity")</error:value>

                   </error:variable>

                 </error:variables>

                 <error:xquery-version>0.9-ml</error:xquery-version>

               </error:frame>

               <error:frame>

                 <error:uri>/eval.xqy</error:uri>

                 <error:line>115</error:line>

                 <error:xquery-version>1.0-ml</error:xquery-version>

               </error:frame>

      </error:stack>

    </error:error></qry:error-trace>

</qry:query-plan>


What setting should i do in database to fasten the query?

Note :- element-range-index is set for the 'Country' element.

Thanks and Regards,
Jaspreet
On Sun, Mar 11, 2012 at 2:45 AM, Jonathan Cook - FM&T <Jonathan.Cook2 at bbc.co.uk<mailto:Jonathan.Cook2 at bbc.co.uk>> wrote:

I've written up a blog post on how to profile your code and the steps I went through.
http://joncook.github.com/blog/2012/02/12/evaluating-mark-logic-xquery-performance/

Jon



-----Original Message-----
From: general-bounces at developer.marklogic.com<mailto:general-bounces at developer.marklogic.com> on behalf of Michael Blakeley
Sent: Tue 28.02.2012 05:47
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] SVC:EXTIME -- Time limit exceeded

First, profile your code to figure out which bit is taking so long. Then rewrite it so it's faster. Repeat until you are happy with performance.

If you post code and profiler output here, you might get some suggestions.

-- Mike

On 27 Feb 2012, at 21:28 , JASPREET SINGH wrote:

> Hi,
>
> I am getting an error SVC:EXTIME -- Time limit exceeded error during search. We have a .NET application that sends request to XDBC server of MarkLogic. We have set the request time out limit as 30 secs which we have to keep anyhow for fast performance.
> But when we search millions of documents inside MarkLogic server, we get the above mentioned error.
>
> What would be the best way to resolve this error? What server settings we should make to optimize our search results?
>
> Thanks and Regards,
> Jaspreet Singh
>
> _______________________________________________
> General mailing list
> General at developer.marklogic.com<mailto:General at developer.marklogic.com>
> http://developer.marklogic.com/mailman/listinfo/general

_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general

_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general


_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://developer.marklogic.com/pipermail/general/attachments/20120313/e77a3f7c/attachment-0001.html 


More information about the General mailing list