[MarkLogic Dev General] CPF pre-commit action?

Geert Josten Geert.Josten at marklogic.com
Fri Jan 15 11:38:52 PST 2016


Have you considered taking an approach similar to Info Studio: insert
unprocessed docs in an separate database in which you have all the CPF
pipelines you need. Not until finishing of processing it gets copied into
the target database. Info Studio was using the Fab database for that
purpose..

Cheers

On 1/15/16, 5:06 PM, "general-bounces at developer.marklogic.com on behalf of
Will Thompson" <general-bounces at developer.marklogic.com on behalf of
wthompson at oconnors.com> wrote:

>Geert,
>
>We're using CPF because some steps may necessitate human intervention, in
>which case they go into a queue, get resolved, a state change occurs, and
>the pipeline marches on. But are you suggesting a pre-commit trigger to
>supplement the existing CPF pipeline?
>
>Danny,
>
>The main issue for us is user experience. For example, in most cases
>document X will already exist in the database and users can see it as
>part of the application. We want to replace document X with an updated
>version and not have it appear visible to users until it has been
>transformed. Hiding it in a collection would hide it from users as well.
>I said "version" so maybe DLS seems like a good fit? But I'm still unsure
>of how to begin the pipeline with the document in a checked-out state.
>
>We do have a solution to this, which is essentially: the application
>takes care of it. But I wanted to see if it were possible to push that
>logic back into CPF and simplify our API such that other applications
>could utilize the pipeline with basic insert and delete operations.
>
>Thanks,
>
>Will
>
>
>> On Jan 15, 2016, at 5:26 AM, Geert Josten <Geert.Josten at marklogic.com>
>>wrote:
>> 
>> Or how about KISS: just use one pre-commit trigger that does all the
>> transformation/processing in one blow..
>> 
>> Kind regards,
>> Geert
>> 
>> On 1/15/16, 12:42 AM, "general-bounces at developer.marklogic.com on behalf
>> of Danny Sokolsky" <general-bounces at developer.marklogic.com on behalf of
>> Danny.Sokolsky at marklogic.com> wrote:
>> 
>>> Hi Will,
>>> 
>>> I can't think of a way to do this without modifying the core cpf code.
>>> For some of the status-change-handling pipeline steps, there are
>>> pre-commit triggers, but those are for cpf bookkeeping mostly, I
>>>believe.
>>> And I would not recommend changing that code.
>>> 
>>> But I would question why you would want to do this.  CPF is designed to
>>> be resilient and to allow you to have multiple steps in your pipeline.
>>> Why not make a step that does the transform its own step?  If you do
>>>not
>>> want the document to be visible until it is transformed, then put it in
>>> some collection initially that is not seen by your application, and
>>>then
>>> take it out of that collection when you are done with the transform (or
>>> something similar to that).
>>> 
>>> Is it just because you want to have one less step in your pipeline that
>>> you want to do this?
>>> 
>>> -Danny
>>> 
>>> -----Original Message-----
>>> From: general-bounces at developer.marklogic.com
>>> [mailto:general-bounces at developer.marklogic.com] On Behalf Of Will
>>> Thompson
>>> Sent: Thursday, January 14, 2016 1:06 PM
>>> To: MarkLogic Developer Discussion
>>> Subject: [MarkLogic Dev General] CPF pre-commit action?
>>> 
>>> Is it possible to configure a CPF pipeline such that when a document is
>>> inserted, transformations are first executed on the document in a
>>> pre-commit stage, and if those complete successfully, then the
>>> transformation result is what's finally committed at that URI? And any
>>> remaining pipeline could carry on without the pre-transformed data ever
>>> being visible to the database?
>>> 
>>> -Will
>>> _______________________________________________
>>> General mailing list
>>> General at developer.marklogic.com
>>> Manage your subscription at:
>>> http://developer.marklogic.com/mailman/listinfo/general
>>> _______________________________________________
>>> General mailing list
>>> General at developer.marklogic.com
>>> Manage your subscription at:
>>> http://developer.marklogic.com/mailman/listinfo/general
>> 
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> Manage your subscription at:
>> http://developer.marklogic.com/mailman/listinfo/general
>
>_______________________________________________
>General mailing list
>General at developer.marklogic.com
>Manage your subscription at:
>http://developer.marklogic.com/mailman/listinfo/general



More information about the General mailing list