[XQZone General] Guaranteed unique?
Howard Katz
howardk at fatdog.com
Thu Dec 8 04:31:02 PST 2005
Jason's solution (pass a "count" param into next-id() to preallocate the
range of id's to be assigned, and only do the xdmp:node-replace() once)
works for me, since in my case it's trivial to determine the count.
Sometimes the simple ideas slip right by you. I shouldn't need to do the
eval() thing as well (tho of course that's looks like a solution too, as you
point out).
Thanks to both,
Howard
> -----Original Message-----
> From: general-bounces at xqzone.marklogic.com
> [mailto:general-bounces at xqzone.marklogic.com] On Behalf Of
> Andy Townsend
> Sent: December 8, 2005 1:54 AM
> To: General XQZone Discussion
> Subject: RE: [XQZone General] Guaranteed unique?
>
>
>
>
> Howard,
>
> Can you not simply use xdmp:eval() rather than
> xdmp:eval-in() to do the
> same thing in the same database.
>
> Also depending on your scenario you could grab blocks of IDs
> in the way
> that Joel suggested - so long as the number of IDs required
> is relatively
> easy to determine. The advantage there would be you could be sure of
> getting consecutive IDs, if that was important.
>
> Here's a variation of the example code from Joel that does both:
>
> define function next-id ($doc-uri,$num-ids as xs:integer) as
> xs:integer {
> let $eval := fn:concat(
> "let $doc-uri :='",$doc-uri,"'
> let $num-ids :=",fn:string($num-ids),"
> let $id := doc($doc-uri)/id
> let $next-val :=
> if ($id castable as xs:integer) then xs:integer($id) + 1
> else 1
> let $next-node := <id>{$next-val + $num-ids - 1}</id>
> let $insert :=
> if ($id) then xdmp:node-replace ($id, $next-node)
> else xdmp:document-insert ($doc-uri, $next-node)
>
> return $next-val")
>
> return xdmp:eval($eval)
> }
> let $seq := (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20)
> let $nextid := next-id("tempuri",fn:count($seq)) - 1
> (: we subtracted 1 from nextid as index starts at 1 in XQuery :)
> return
> for $x at $index in $seq
> return $index + $nextid
>
>
> Andy
> |-----------------------------+------------------------------
> -------------|
> | "Howard Katz" |
> |
> | <howardk at fatdog.com> |
> |
> | Sent by: |
> To|
> | general-bounces at xqzone.mar|
> "'Gen|
> | klogic.com |
> eral |
> | |
> XQZon|
> | 08/12/2005 01:43 |
> e |
> | |
> Discu|
> | Please respond to |
> ssion|
> | General XQZone |
> '" |
> | Discussion |
> <gene|
> | <general at xqzone.marklog|
> ral at x|
> | ic.com> |
> qzone|
> | |
> .mark|
> | |
> logic|
> | |
> .com>|
> | |
> cc|
> | |
> |
> | |
> Subject|
> | |
> RE: |
> | |
> [XQZo|
> | |
> ne |
> | |
> Gener|
> | |
> al] |
> | |
> Guara|
> | |
> nteed|
> | |
> uniqu|
> | |
> e? |
> | |
> |
> | |
> |
> | |
> |
> | |
> |
> | |
> |
> | |
> |
> |-----------------------------+------------------------------
> -------------|
>
>
>
>
>
>
>
> I'm trying to do something similar and am encountering the
> samed dreaded
> XDMP-CONFLICTINGUPDATES message. I'm calling next-id( ) from
> within a "for"
> loop. Is it that static analysis of the query is seeing multiple
> xdmp:replace-nodes() -- tho they're actually invoked
> sequentially -- and
> thus throws this error?
>
> Unfortunately I don't have the luxury of doing an eval-in()
> in another
> database. Are there any other workarounds within the same
> db? I can see
> grabbing all the id's in the document, doing an appropriate
> "order by", and
> assigning the largest value + 1 as the next value to be
> assigned, but that
> doesn't seem like a lot of fun for a large document.
>
> Howard
>
> > -----Original Message-----
> > From: general-bounces at xqzone.marklogic.com
> > [mailto:general-bounces at xqzone.marklogic.com] On Behalf Of
> > Stewart, Joel (LNG-DAY)
> > Sent: November 28, 2005 8:02 AM
> > To: General XQZone Discussion
> > Subject: RE: [XQZone General] Guaranteed unique?
> >
> > I have been wondering how to accomplish this when needing to
> > call it a
> > number of times from within an xquery function, e.g.:
> >
> > define function next-id ($doc-uri) as xs:integer {
> > let $id := doc($doc-uri)/id
> > let $next-val :=
> > if ($id castable as xs:integer) then xs:integer
> > ($id) + 1
> > else 1
> > let $next-node := <id>{$next-val}</id>
> > let $insert :=
> > if ($id) then xdmp:node-replace ($id, $next-node)
> > else xdmp:document-insert ($doc-uri, $next-node)
> >
> > return $next-val
> > }
> >
> > let $seq := (1,2,3,4,5,6,7,8,9)
> > return
> > for $x in $seq
> > return next-id("tempuri")
> >
> > running this will result in
> > XDMP-CONFLICTINGUPDATES: Conflicting updates
> > xdmp:document-insert("tempuri", /id, (), (), 0) and
> > xdmp:document-insert("tempuri", /id, (), (), 0) at line 18
> >
> >
> > If I was to call this from a Java layer that iterates the
> nodes to be
> > inserted, I recognize this method would work such that you
> > either get a
> > number or you get an exception. Retry logic would need to
> > be written.
> >
> > We put a lot of work in XQuery modules, and iterate large
> numbers of
> > nodes and do many inserts within the xquery function for
> performance.
> > To my knowledge, we have not needed an auto-increment
> function in all
> > this, but I have often wondered what we should do should that need
> > arise. Hopefully, the number of ids that will be needed can be
> > determined easily and quickly before invoking the functions
> > that do the
> > inserts. In this way, we can pass in a sequence or range of
> > ids. But
> > if the count of nodes is hard or expensive to determine - I
> > was trying
> > to think of a work around.
> >
> > One idea is to use eval-in. I have been wondering if using
> > that to put
> > your ids in another database would work around the above issue.
> > Apparently it does. By using eval-in, the "next-id" document is
> > actually committed. So trying the following will work if
> you crate a
> > new DB to actually execute the "next-id" function. The
> code could be
> > more pretty by putting next-id in a module in that db, but
> > it gets the
> > idea across.
> >
> > define function next-id ($doc-uri) {
> > let $func :=
> > fn:concat("
> > let $doc-uri :=""",$doc-uri,"""
> > let $id := doc($doc-uri)/id
> > let $next-val :=
> > if ($id castable as xs:integer) then xs:integer
> > ($id) + 1
> > else 1
> > let $next-node := <id>{$next-val}</id>
> > let $insert :=
> > if ($id) then xdmp:node-replace ($id, $next-node)
> > else xdmp:document-insert ($doc-uri, $next-node)
> >
> > return $next-val
> > ")
> >
> > return xdmp:eval-in($func, xdmp:database("ANOTHER-DB"))
> >
> > }
> >
> > let $seq := (1,2,3,4,5,6,7,8,9)
> > return
> > for $x in $seq
> > return next-id("tempuri")
> >
> > So, running this will work. The hard part will be putting error
> > handling and retry logic in the function calling it - if it
> > so happens
> > two threads get on that node at the same time and cause conflicting
> > updates. But I think this goes a long way toward what is
> > needed to be
> > implemented natively in marklogic - some "autoincrement"
> > function that
> > is not bound to the same rollback, and is atomic such that
> > you will not
> > get the conflicting updates exception.
> >
> > Other ideas are welcomed.
> >
> > Thanks.
> > joel
> >
> > -----Original Message-----
> > From: general-bounces at xqzone.marklogic.com
> > [mailto:general-bounces at xqzone.marklogic.com] On Behalf Of
> > Ron Hitchens
> > Sent: Tuesday, November 22, 2005 7:40 PM
> > To: General XQZone Discussion
> > Subject: Re: [XQZone General] Guaranteed unique?
> >
> >
> > Here is a function to do what Ian describes, lifted
> > from the xfaqtor code available on xqzone:
> >
> >
> > (: Return a monotonically increasing integer (xs:integer) value.
> > Store the value in the root node of doc ($doc-uri).
> > You could enhance this file to store IDs for multiple uses.
> > Logic is to fetch the current id, add one if it exists
> > or assign the
> > value to one if it doesn't yet exist, then replace or
> > insert the new
> > value before returning it.
> > :)
> > define function next-id ($doc-uri) as xs:integer
> > {
> > let $id := doc($doc-uri)/id
> > let $next-val :=
> > if ($id castable as xs:integer) then xs:integer
> > ($id) + 1
> > else 1
> > let $next-node := <id>{$next-val}</id>
> > let $insert :=
> > if ($id) then xdmp:node-replace ($id, $next-node)
> > else xdmp:document-insert ($doc-uri, $next-node)
> >
> > return $next-val
> > }
> >
> >
> > On Nov 22, 2005, at 4:15 PM, Ian Small wrote:
> >
> > > Andy -
> > >
> > > Just create a document that contains the serial number in
> > > question. Write some simple XQuery to retrieve the id from the
> > > document, increment it, and update the document (using xdmp:node-
> > > replace(), etc.).
> > >
> > > Because your XQuery module is performing an "update", it
> > will block
> > > all other requests trying to update that same document, thereby
> > > guaranteeing unique IDs.
> > >
> > > Only "queries" (ie. requests which are statically analyzed to be
> > > "read-only" requests) run lock-free per your description.
> > Updates
> > > (ie. requests which are statically analyzed to potentially change
> > > the state of the database) lock documents and consequently block
> > > each other as appropriate.
> > >
> > > ian
> > >
> > > -----Original Message-----
> > > From: general-bounces at xqzone.marklogic.com [mailto:general-
> > > bounces at xqzone.marklogic.com] On Behalf Of Andy Townsend
> > > Sent: Tuesday, November 22, 2005 1:43 PM
> > > To: general at xqzone.marklogic.com
> > > Subject: [XQZone General] Guaranteed unique?
> > >
> > >
> > >
> > >
> > > Anyone know a nice way to generate guaranteed unique
> > sequential IDs
> > > with XQuery in Mark Logic?
> > >
> > > So I can for example create the values for: id="1" ,
> > id="2", etc...
> > > and be sure that two users cannot ever pick up the same number.
> > > Just having a document in the database with the current number
> > > doesn't seem enough to me because I believe there would be a race
> > > condition where two users could start at about the same
> > time and so
> > > see the same initial 'state' of the database which I presume the
> > > XQuery would insist they both saw throughout their queries.
> > >
> > > I thought that document-locks might get me there (though in
> > > probably contravention of the standard) but have not had any joy
> > > with those as yet
> > >
> > > In Java it would be easy - just a serialised block - equally easy
> > > in C++, but what about XQuery?
> > >
> > > Someone must have done this already! :-)
> > >
> > > Thanks
> > >
> > > Andy
> > >
> > >
> > #############################################################
> > #########
> > > The information contained in this e-mail and any subsequent
> > > correspondence is private and confidential and intended
> > solely for
> > > the named recipient(s). If you are not a named recipient,
> > you must
> > > not copy, distribute, or disseminate the information, open any
> > > attachment, or take any action in reliance on it. If you have
> > > received the e-mail in error, please notify the sender and delete
> > > the e-mail.
> > >
> > > Any views or opinions expressed in this e-mail are those of the
> > > individual sender, unless otherwise stated. Although this e-mail
> > > has been scanned for viruses you should rely on your own virus
> > > check, as the sender accepts no liability for any damage arising
> > > out of any bug or virus infection.
> > >
> > #############################################################
> > #########
> > > _______________________________________________
> > > General mailing list
> > > General at xqzone.marklogic.com
> > > http://xqzone.com/mailman/listinfo/general
> > > _______________________________________________
> > > General mailing list
> > > General at xqzone.marklogic.com
> > > http://xqzone.com/mailman/listinfo/general
> >
> > ---
> > Ron Hitchens {ron.hitchens at marklogic.com} 650-655-2351
> >
> >
> > _______________________________________________
> > General mailing list
> > General at xqzone.marklogic.com
> > http://xqzone.com/mailman/listinfo/general
> > _______________________________________________
> > General mailing list
> > General at xqzone.marklogic.com
> > http://xqzone.com/mailman/listinfo/general
> >
>
> _______________________________________________
> General mailing list
> General at xqzone.marklogic.com
> http://xqzone.com/mailman/listinfo/general
>
>
> #############################################################
> #########
> The information contained in this e-mail and any subsequent
> correspondence is private and confidential and intended solely
> for the named recipient(s). If you are not a named recipient,
> you must not copy, distribute, or disseminate the information,
> open any attachment, or take any action in reliance on it. If you
> have received the e-mail in error, please notify the sender
> and delete
> the e-mail.
>
> Any views or opinions expressed in this e-mail are those of the
> individual sender, unless otherwise stated. Although this
> e-mail has
> been scanned for viruses you should rely on your own virus check, as
> the sender accepts no liability for any damage arising out
> of any bug
> or virus infection.
> #############################################################
> #########
> _______________________________________________
> General mailing list
> General at xqzone.marklogic.com
> http://xqzone.com/mailman/listinfo/general
>
More information about the General
mailing list