[MarkLogic Dev General] Optimizing lookups
Runstein, Robert E. (Contr) (IS)
Robert.Runstein at ngc.com
Mon Nov 23 09:41:14 PST 2009
cq profiling is *real* helpful and pointed out something interesting.
My application utilizes lookup documents containing key/value pairs of
<entry><code>abc</code><name>Aay Bee Cee</name><entry>
There are about 6000 entries one of the documents. When accessed as
fn:doc("lookup.xml")/document/entry[code = $code]/name
profiling shows 6000 code = $code tests are executed. Since there are
hundreds of lookups while processing a single document, a significant
amount of time is devoted to this test.
I've run xdmp:query-trace which indicates that the path is fully
searchable and the predicate does not use indexes, but does use a hash
I've tried adding an element range index to the code element, but that
did not change the trace.
As a workaround I've been able to significantly reduce processing time
by loading the lookup document entry nodes into a map keyed by the code
value and declaring that map as a variable at the top of my module, but
I'm concerned that this map gets loaded only at server startup and any
updates to the lookup.xml document would require a server reboot. While
I could code an "init" function to reload the maps I'd rather avoid this
complexity if there is a built in way that I can avoid the 6000 tests
per value resolution.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the General