[MarkLogic Dev General] Query Speed

Irvine, Joseph [USA] Irvine_Joseph at bah.com
Fri Aug 10 13:48:57 PDT 2012


Hello,

The following query, for example, with roughly 6500 records took 55.34 seconds according to the MarkLogic performance monitor. 

I changed the names and scenario, but used the same exact template. Assume in this case that I have 6500 records. Each records the name of the fruit, and how many days it took to get from the field to the supermarket. This query, then, goes through and gets a unique list of the fruits, since there are repeats. Bananas show up in one record as taking 5 days to get to Destination A, while Bananas show up as taking 12 days to get to Destination B. In this query, I want to get a list of all of the fruits represented as having been transported in the 6500 records, and then list the average time that it took for that fruit. We might find for example that Bananas get to the locations quicker because they are air-freighted while Watermelons take longer since they are shipped via ground transportation. That is basically what we are looking for.

QUERY:

for $d in distinct-values(/Data/fruitName) 
return 
<item>
	<fruitName>{$d}</fruitName>
	<averageFrequency>{avg(/qdaData/daysToMarket[/qdaData/fruitName = $d])}</averageFrequency>
</item>

That seems highly excessive to me. Each record is composed of just 30 XML fields. All 6500 records take up less than 30mb disk space cumulatively. The server I am on is running 24gb RAM. I have all of the fields above as indexed database elements within MarkLogic. Pretend the following is a rough version of one of the XML files:

<Data>
	<fruitName>Banana</fruitName>
	<daysToMarket>5</daysToMarket>
	<finalDestination>Phoenix</finalDestination>
	<origin>Argentina</origin>
	<weight>5.5</weight>
	... etc
</Data>

Thanks,
Joseph Irvine


-----Original Message-----
From: general-bounces at developer.marklogic.com [mailto:general-bounces at developer.marklogic.com] On Behalf Of Michael Blakeley
Sent: Friday, August 10, 2012 1:13 PM
To: MarkLogic Developer Discussion
Subject: [External] Re: [MarkLogic Dev General] Query Speed

It sounds likely that the .NET queries are not quite the same as the AB queries. If you can identify a slow query, post it to the list and you may get some useful tuning advice.

-- Mike

On 10 Aug 2012, at 09:52 , Irvine, Joseph [USA] wrote:

> Hello,
>  
> I have a basic .NET client that connects to a MarkLogic database with XCC.
>  
> I built it around a database that had 1000 XML documents ingested at the time. The queries were quick, almost instantaneous at the time, and the program ran smoothly as a result. I have since increased the size of the database to 20,000 XML documents. While the forms I have through the Application Builder still respond incredibly quickly, the .NET queries are incredible slow, some taking several minutes to run, as opposed to the near instantaneous speed I had before.
>  
> Is there some way to increase the speed for external queries?
>  
> Thank you,
> Joseph Irvine
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general

_______________________________________________
General mailing list
General at developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general


More information about the General mailing list