[MarkLogic Dev General] xqsync / cpf incompatibility
sokolov at ifactory.com
Fri Mar 27 10:05:17 PST 2009
I had sent the following report to support at marklogic.com; they indicated that since xqsync is unsupported this mailing list would be a better place to discuss the problem. Currently I don't have an issue, since I have a good workaround, but I thought others might be interested to know about this, so for posterity's sake:
I don't know if you would consider this a bug, strictly speaking, but it is some really weird and annoying behavior that cost me a day to track
down and deal with, so it's probably worth at least a note in the documentation somewhere, or at the very least preservation on this list.
I was having a problem with a cpf pipeline: it would only operate on about 1/4-1/3 of the documents I was loading. The number fluctuated unpredictably each time I would try loading the same documents, and it didn't seem to have anything to do with the documents themselves, since different docs would succeed / fail each time.After turning on tracing, I noticed in the logs a variety of different messages: something about checking a transition and skipping a document that sounded suspicious, and then a bunch of deadlocks detected as well.
Anyway, I'll spare you all the detective work, but in the end I tracked this down to xqsync - apparently the way it is setting document properties during loading is incompatible with cpf. When I disabled its property-loading behavior, everything worked fine.
My suspicion is that xqsync is updating properties after the document is initially created and causing a race condition with cpf, which is doing the same thing. I'd suggest changing xqsync, if possible, to set the properties during the document creation, or possibly wrapping a transaction around both operations in some way. If that can't be done then perhaps it's just best to document this in the README.
More information about the General