[MarkLogic Dev General] more record loader questions

Alan Darnell alan.darnell at utoronto.ca
Thu Oct 30 14:00:42 PST 2008


We've been running into some problems using record loader with the  
property

CONTENT_FACTORY_CLASSNAME =  
com.marklogic.recordloader.xcc.XccModuleContentFactory

and

CONTENT_MODULE_URI = /path-to-our/load.xqy

Load.xqy invokes another module which runs an XQuery transformation on  
the input document -- load.xqy then inserts the transformed document  
into ML.

We think that record loader is identifying all 450,000 records in our  
test because we see messages such as:

Oct 30, 2008 5:29:26 PM com.marklogic.recordloader.Monitor monitor
INFO: inserted record 306518 as /01406736/ 
v75inone_s918/547_psoltm11.xml (179 tps, 763 kB/s), with 0 error(s)

But only about 90,000 of the records end up in our ML database.

There are no error messages showing up in the ML server error log to  
account for the difference and nothing is showing up in  
simplelogger.log.

Is there a way to send errors up from "load.xqy" and the modules it  
invokes to record_loader so that we can see what's happening in  
simplelogger.log?  Can we change the logging in record loader so  
instead of reporting periodically on the records loaded, it reports a  
status for each record loaded?

Alan

  


More information about the General mailing list