[MarkLogic Dev General] How to improve performance of update query?

Michael Blakeley mike at blakeley.com
Fri Aug 31 08:47:48 PDT 2012


No offense, I hope, but... what you are doing now won't scale. You are already seeing that.

When you find yourself building your own update stack using maps, and calling xdmp:eval to process it... you're probably digging yourself into a hole. Instead of fighting with the database, try to take advantage of its strengths. One of its core strengths is handling lots of documents. Handling a few really large documents is possible, but not optimal.

I don't know much about your application, but with typical document sizes you can plan on forests with 4-16M documents each. Refining that range requires more information about indexes and queries. Once you know that, and know how many documents you need to retain, and you can decide how many forests you will need. That will tell you what hardware you need.

-- Mike

On 31 Aug 2012, at 01:41 , Tomo Simeonov wrote:

> Yes, this could help, but how well does it scale? What if I am going to store millions of objects per day? Isn't that going to cause problems like huge indexes?
> 
> Tomo
> 
> Sent from Yahoo! Mail on Android
> 



More information about the General mailing list