[MarkLogic Dev General] Totally odd search:search behavior

Shannon Shiflett at virginia.edu
Wed Aug 11 10:58:01 PDT 2010


Micah, thanks a lot. I'm a fan of the debug option, but my mistake was in assuming Search API is the culprit :-) I'll check on that, get more information, and if the solution doesn't jump out at me, take it to Support. Have a great day,
Shannon

On Aug 11, 2010, at 1:44 PM, Micah Dubinko wrote:

> Shannon,
> 
> Include the option <debug>true</debug>, and you will get some additional output, including the full flwor that gets used to run the search. Then you can determine
> 
> 1) Does the flwor (including cts:search) look correct?
> 2) Does the flowr return the expected results when run in CQ or a similar environment?
> 
> Those will help narrow down the search aspect, and determine whether the problem is happening in Search API or not. Going through support is also an option; they are good at asking the standard set of questions (like what version of the server are you running?; are you using any fragmentation?; what do your data look like?; etc.
> 
> Thanks! -m
> 
> On Aug 11, 2010, at 9:34 AM, Shannon wrote:
> 
>> Hi again,
>> 
>> First, I realize in my previous post (which has not been posted yet and is awaiting moderator's approval because it contains images--hope it gets approved soon...) I was not actually passing $options to search:search, sorry -- although it does not affect the point of my note -- but now I am and I am getting extremely weird results in a particular search:search case.
>> 
>> I have outlined 4 cases below. The first 3 return the correct results. I have verified the counts, the number of results is accurate and the snippets look good.
>> 
>> However, case 4 drives me into a ditch for two reasons: @total should be 30, not 29 (it is omitting document LBJ-0545.xml entirely even though it is retrieved in the other cases and it definitely contains the search term), and while @total is 29, only 17 results are actually returned (I have pasted the output below--the highest @index is 17).
>> 
>> Can anyone demystify #4 for me because it looks very, very wrong and is missing a document which is unsettling.
>> 
>> So, search:search by itself from Search API is returning incorrect results, and search:search within AppBuilder is returning incorrect snippets. Could really use some support here guys, thanks.
>> 
>> 1 --
>> 
>> xquery version "1.0-ml";
>> import module namespace search = "http://marklogic.com/appservices/search"
>>   at "/MarkLogic/appservices/search/search.xqy";
>> 
>> let $options :=
>> <options xmlns="http://marklogic.com/appservices/search">
>> <searchable-expression>/tei:TEI/tei:text/tei:body/tei:div</searchable-expression>
>> <term>
>>  <term-option>case-insensitive</term-option>
>> </term>
>> </options>
>> return
>> search:search("Peace", $options, 1, 30)
>> 
>> =>
>> 
>> <search:response total="30" ...
>> 
>> 2 --
>> 
>> xquery version "1.0-ml";
>> import module namespace search = "http://marklogic.com/appservices/search"
>>   at "/MarkLogic/appservices/search/search.xqy";
>> 
>> let $options :=
>> <options xmlns="http://marklogic.com/appservices/search">
>> <searchable-expression>/tei:TEI/tei:text/tei:body/tei:div</searchable-expression>
>> <term>
>>  <term-option>case-insensitive</term-option>
>> </term>
>> </options>
>> return
>> search:search("peace", $options, 1, 30)
>> 
>> =>
>> 
>> <search:response total="30" ...
>> 
>> 3 --
>> 
>> xquery version "1.0-ml";
>> import module namespace search = "http://marklogic.com/appservices/search"
>>   at "/MarkLogic/appservices/search/search.xqy";
>> 
>> let $options :=
>> <options xmlns="http://marklogic.com/appservices/search">
>> <searchable-expression>/tei:TEI/tei:text/tei:body/tei:div</searchable-expression>
>> <term>
>>  <term-option>case-sensitive</term-option>
>> </term>
>> </options>
>> return
>> search:search("Peace", $options, 1, 30)
>> 
>> =>
>> 
>> <search:response total="15" ...
>> 
>> 4 --
>> 
>> xquery version "1.0-ml";
>> import module namespace search = "http://marklogic.com/appservices/search"
>>   at "/MarkLogic/appservices/search/search.xqy";
>> 
>> let $options :=
>> <options xmlns="http://marklogic.com/appservices/search">
>> <searchable-expression>/tei:TEI/tei:text/tei:body/tei:div</searchable-expression>
>> <term>
>>  <term-option>case-sensitive</term-option>
>> </term>
>> </options>
>> return
>> search:search("peace", $options, 1, 30)
>> 
>> =>
>> 
>> <search:response total="29" start="1" page-length="30" xmlns:search="http://marklogic.com/appservices/search">
>> <search:result index="1" uri="LBJ-2660.xml" path="fn:doc(&quot;LBJ-2660.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div[2]" score="156" confidence="0.405106" fitness="0.58184">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2660.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div[2]/tei:sp[7]/tei:p">...land. Under our system of government, local authorities have the basic responsibility for civil <search:highlight>peace</search:highlight>...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="2" uri="LBJ-0744.xml" path="fn:doc(&quot;LBJ-0744.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="156" confidence="0.405106" fitness="0.58184">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0744.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[228]/tei:p/tei:note">...and to maintaining law and order, emphasizing the responsibilities of local officials for "civil <search:highlight>peace</search:highlight>." He ended his statement on this matter by explaining, "a Federal police force...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="3" uri="LBJ-2800.xml" path="fn:doc(&quot;LBJ-2800.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="130" confidence="0.369809" fitness="0.531145">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2800.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[60]/tei:p">...the sick and the lame and the old and the poor people and education and <search:highlight>peace</search:highlight>.  And <search:highlight>peace</search:highlight> in the world, and you've got to do something besides talk about it....</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="4" uri="LBJ-0602.xml" path="fn:doc(&quot;LBJ-0602.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="104" confidence="0.330768" fitness="0.475071">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0602.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[23]/tei:p[1]">...told me that he wanted me to assure you
>>                           that there would be complete <search:highlight>peace</search:highlight> and order, law and order, in St.
>>                           Helena Parish. I called you...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="5" uri="LBJ-2799.xml" path="fn:doc(&quot;LBJ-2799.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="104" confidence="0.330768" fitness="0.475071">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2799.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[55]/tei:p">...mudslinging. They want to talk bread-and-butter issues. They want to talk about
>>                           <search:highlight>peace</search:highlight> and war. They want to talk about nuclear holy-caust </search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="6" uri="LBJ-2668.xml" path="fn:doc(&quot;LBJ-2668.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="104" confidence="0.330768" fitness="0.475071">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2668.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[36]/tei:p">...to the actual vote in November, they'll be more concerned first about the <search:highlight>peace</search:highlight>. They'll be concerned
>>                           about the prosperity. They'll be concerned about the...</search:match>
>>     <search:match path="fn:doc(&quot;LBJ-2668.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[37]/tei:p/tei:note">...let alone the Presidency, are a disservice to our national security, a disservice to <search:highlight>peace</search:highlight>, and, as for that matter, a great disservice to the entire free world....</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="7" uri="LBJ-2662.xml" path="fn:doc(&quot;LBJ-2662.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="78" confidence="0.286453" fitness="0.411423">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2662.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[2]/tei:p">...people. He said that he had been assured by everybody that he'll have complete <search:highlight>peace</search:highlight> and order in that parish. He issued a statement and said that you asked...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="8" uri="LBJ-0638.xml" path="fn:doc(&quot;LBJ-0638.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="78" confidence="0.286453" fitness="0.411423">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0638.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[33]/tei:p">...advise with you some, because I see you stand up in meetings and speak your <search:highlight>peace</search:highlight>, and it's always on the side of the folks.</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="9" uri="LBJ-0587.xml" path="fn:doc(&quot;LBJ-0587.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="78" confidence="0.286453" fitness="0.411423">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0587.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[11]/tei:p[3]"> was in on it. The justice of the <search:highlight>peace</search:highlight> was in on it. And there were seven other men. We have all those...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="10" uri="LBJ-2532.xml" path="fn:doc(&quot;LBJ-2532.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2532.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:incident/tei:desc">...         brief fragment of office conversation, with McGeorge Bundy saying “only
>>                       firmness makes <search:highlight>peace</search:highlight>.” President Johnson asks the operator to keep Senator
>>            ...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="11" uri="LBJ-2646.xml" path="fn:doc(&quot;LBJ-2646.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div[1]" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2646.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div[1]/tei:sp[132]/tei:p">And I hope that your trip is going to mean serenity and <search:highlight>peace</search:highlight> and just a
>>                           revitalization. And I know you’ve gone through years of just...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="12" uri="LBJ-2802.xml" path="fn:doc(&quot;LBJ-2802.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2802.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[8]/tei:p[4]">...morals and confidence in the administration of the law and of people’s right to <search:highlight>peace</search:highlight> and security on
>>                           the streets and in their homes, and that you want the...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="13" uri="LBJ-2700.xml" path="fn:doc(&quot;LBJ-2700.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2700.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[32]/tei:p[3]">...judicious fellow. He’s a pretty careful fellow. He’s
>>                           pretty strong on <search:highlight>peace</search:highlight>. That’s showing up in all of our polls. By
>>                           contrast, he...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="14" uri="LBJ-2670.xml" path="fn:doc(&quot;LBJ-2670.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-2670.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[13]/tei:p/tei:note">The President had spent much of the summer trying to keep the <search:highlight>peace</search:highlight> in Cyprus in a conflict between Greeks and Turks on the island. Early this...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="15" uri="LBJ-0601.xml" path="fn:doc(&quot;LBJ-0601.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0601.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[70]/tei:p">...number one in every poll we've taken--the number one
>>                           problem--is not <search:highlight>peace</search:highlight>. It's civil rights.</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="16" uri="LBJ-0565.xml" path="fn:doc(&quot;LBJ-0565.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0565.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[59]/tei:p">...get sick, what it's going to mean economically. When the country's at <search:highlight>peace</search:highlight>, as it is now, and they're not concerned about the Russians as much...</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:result index="17" uri="LBJ-0666.xml" path="fn:doc(&quot;LBJ-0666.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div" score="52" confidence="0.233888" fitness="0.335926">
>>   <search:snippet>
>>     <search:match path="fn:doc(&quot;LBJ-0666.xml&quot;)/tei:TEI/tei:text/tei:body/tei:div/tei:sp[79]/tei:p[2]">...to encourage a backlash. That's where his future is; it's not in <search:highlight>peace</search:highlight> and harmony.</search:match>
>>   </search:snippet>
>> </search:result>
>> <search:qtext>peace</search:qtext>
>> <search:metrics>
>>   <search:query-resolution-time>PT0.005648S</search:query-resolution-time>
>>   <search:facet-resolution-time>PT0.000047S</search:facet-resolution-time>
>>   <search:snippet-resolution-time>PT0.09151S</search:snippet-resolution-time>
>>   <search:total-time>PT0.115786S</search:total-time>
>> </search:metrics>
>> </search:response>
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> http://developer.marklogic.com/mailman/listinfo/general
> 
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general



More information about the General mailing list