[MarkLogic Dev General] PipelinequestionandXDMP-CONFLICTINGUPDATES error

[Tim Meagher]

Hi Geert,

 

I reckon that's a good question.  I'm merely using the Content Processing
Framework to create the two pipelines and I've assumed these use post-commit
triggers.  Even so, there are two distinct pipelines (and hence two distinct
transactions), the first pipeline which inserts doc B in URI-2/B, and the
second which uses the insertion of doc B in URI-2/B as a trigger to initiate
the second pipeline.

 

I don't expect the second pipeline to be able to start processing doc B
until it has been inserted, which I expect will only occur after the
transaction associated with the first pipeline has completed, whether it is
spawned (post-commit) or invoked within the same triggering transaction
(pre-commit), so whether or not the first pipeline uses a pre-or post-commit
trigger, I can't see why there would be any conflict.

 

At this point I don't have a lot of confidence in my dual pipeline design.
Without nailing this down I'm inclined to try to wrap the processing that
takes place in these two pipelines into one pipeline.  I was using the
second pipeline as what I thought would be a simple way to ensure that there
would be no conflict deleting doc A since the first pipeline should have
completed processing before beginning to process doc B in the context of the
second pipeline.

 

In fact I'm not certain from the error log dump if the problem occurs
because there is a conflict with updating doc B or trying to delete doc A.
In either case, the problem is not repeatable when I try to re-insert or
update doc A in URI-1 (which is the trigger for the first pipeline).  BTW,
there are a couple thousand documents that are inserted into the triggering
directory of the first pipeline, so there is a decent load on the system,
and I don't think I'm overflowing the task queue.

 

Tim

 

-----Original Message-----
From: general-bounces at developer.marklogic.com
[mailto:general-bounces at developer.marklogic.com] On Behalf Of Geert Josten
Sent: Monday, November 16, 2009 3:12 AM
To: General Mark Logic Developer Discussion
Subject: RE: [MarkLogic Dev General]
PipelinequestionandXDMP-CONFLICTINGUPDATES error

 

Hi Tim,

 

Could it be that you are using pre-commit triggers? Then it could be that
all updates are treated as part of a single transaction. If so, revert to
post-commit triggers..

 

Kind regards,

Geert

 

> 

 

 

Drs. G.P.H. Josten

Consultant

 

 

http://www.daidalos.nl/

Daidalos BV

Source of Innovation

Hoekeindsehof 1-4

2665 JZ Bleiswijk

Tel.: +31 (0) 10 850 1200

Fax: +31 (0) 10 850 1199

http://www.daidalos.nl/

KvK 27164984

De informatie - verzonden in of met dit emailbericht - is afkomstig van
Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit
bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan
dit bericht kunnen geen rechten worden ontleend.

 

 

> From: general-bounces at developer.marklogic.com

> [mailto:general-bounces at developer.marklogic.com] On Behalf Of

> Tim Meagher

> Sent: zondag 15 november 2009 0:23

> To: 'General Mark Logic Developer Discussion'

> Subject: RE: [MarkLogic Dev General] Pipeline question

> andXDMP-CONFLICTINGUPDATES error

> 

> I should also add that I have a delete trigger such that if

> doc B is deleted from /URI-2/B, then doc C is also deleted

> from /URI-3/C.

> 

> 

> 

> - Tim

> 

> ________________________________

> 

> From: general-bounces at developer.marklogic.com

> [mailto:general-bounces at developer.marklogic.com] On Behalf Of

> Tim Meagher

> Sent: Saturday, November 14, 2009 2:50 PM

> To: 'General Mark Logic Developer Discussion'

> Subject: [MarkLogic Dev General] Pipeline question

> andXDMP-CONFLICTINGUPDATES error

> 

> 

> 

> Hi Folks,

> 

> 

> 

> In have researched previous postings that indicate that the

> XDMP-CONFLICTINGUPDATES error can be related to deleting and

> updating a document, but I'm wondering if it can also be

> related to a document being processed by multiple pipelines.

> 

> 

> 

> I have 2 pipelines, P1 and P2.  P1 works like this:

> 

> 

> 

> When doc A is inserted into /URI-1/A, it triggers P1 which

> creates doc B and inserts it into /URI-2/B.

> 

> 

> 

> When doc B is inserted into /URI-2/B, it triggers P2 which

> creates doc C and inserts it into /URI-3/C.  The last thing

> that P2 does is to delete doc A from /URI-1/A.

> 

> 

> 

> Note that /URI-1/A and /URI-2/B are in the same database, but

> /URI-3/C gets inserted into a different database via using

> xdmp:invoke.

> 

> 

> 

> Also note that the error only occurs when doc A is inserted

> into /URI-1/A for the first time, even though both pipelines

> are configured to run when either new or updated are

> inserted.  Subsequent updates do not appear to be causing the

> XDMP-CONFLICTINGUPDATES error.  I addition, the

> XDMP-CONFLICTINGUPDATES error does not occur for all

> documents, but maybe only 5% of the docs that are inserted.

> 

> 

> 

> So my question is, is it possible that there can be a race

> condition between P1 and P2 in that P1 has completed

> inserting B into the database which triggers P2 before P1 has

> fully updated the status of the document B?

> 

> 

> 

> Thank you!

> 

> 

> 

> Tim Meagher

> 

> 

> 

> 

 

_______________________________________________

General mailing list

General at developer.marklogic.com

http://xqzone.com/mailman/listinfo/general

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://xqzone.marklogic.com/pipermail/general/attachments/20091116/65693fbd/attachment-0001.html