[MarkLogic Dev General] How to improve performance of update query?

Tomo Simeonov tomosimeonov at yahoo.com
Thu Aug 30 12:02:01 PDT 2012


Hi,

I have been working on a query for updating elements in the database.
I am providing many values and then  for the elements that have any of this values I am doing node-replace. Those elements can be in any of the documents in the database not in just one.
It doesn't run very fast if I want to update more than 100 elements in one go and this is just a test, later one a could end up updating thousands.

What I am doing 
1) Search for elements that have one of those values - it has good performance as it uses indexes 
2) I am creating map in which I am storing maps - each stored map contains the nodes that need to be changed in one specific document
3) For each document to be updated I am creating separate transaction with xdmp:eval and provide the correct map, in the transaction I am doing the node-replaces as I do not want to lock all the documents I am searching through.

This works almost perfectly until xdmp:eval part is reache. All evals are executed in serial not in parallel which is causing the bad performance.
My question is - is there a way to execute does dmp:evals in parallel ( xdmp:spawn is not  an option for now)?
Also is there  a better way to do many updates to nodes across many documents? 

Thank you, 
Tomo 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://developer.marklogic.com/pipermail/general/attachments/20120830/e9c541ef/attachment.html 


More information about the General mailing list